$\def\pa{\mathsf{PA}}\def\zff{\mathsf{ZF_{fin}}}\def\zffm{\mathsf{ZF_{fin}^-}}$Your theory (let me denote it $T$ for the moment) is mutually interpretable with $\pa$, but it is not bi-interpretable with $\pa$, and a fortiori not synonymous.
On the one hand, it is easy to see that $\pa$ interprets $T$ using Ackermann’s interpretation ($n\in x$ iff the $n$th bit in the binary expansion of $x$ is $1$), as mentioned in Joel’s answer.
For an interpretation of $\pa$ in $T$, it is easiest to make a short detour through classical results (by now folklore) on finite set theory, due to Mycielski, Vopěnka, and Sochor (apparently). $\pa$ is bi-interpretable with the theory $\zff$ of finite sets, which can be axiomatized by
Extensionality
Existence of $\varnothing$
Existence of $x\cup\{y\}$ for all $x$ and $y$
Induction: $\phi(\varnothing)\land\forall x,y\,\bigl(\phi(x)\to\phi(x\cup\{y\})\bigr)\to\forall x\,\phi(x)$
$\in$-induction: $\forall x\,\bigl(\forall y\in x\,\phi(y)\to\phi(x)\bigr)\to\forall x\,\phi(x)$
Moreover, the theory $\zffm$ axiomatized by 1–4 interprets $\zff$ (hence $\pa$), namely it proves axioms of $\zff$ relativized to the class WF of hereditarily well-founded sets: $x$ is in WF iff there is a transitive set $y\supseteq x$ such that every nonempty subset of $y$ has an $\in$-minimal element. (It is a longish but straightforward exercise to show that $\zffm$ proves all the usual axioms of $\def\zfc{\mathsf{ZFC}}\zfc$ except infinity and foundation; some of these may be useful for verification of this interpretation.) (NB: In recent literature stemming from a rediscovery of some of these results, the notation $\zff$ is used for a weaker theory that only has $\zfc$-style foundation axiom in place of $\in$-induction, and our $\zff$ is denoted $\zff+\mathsf{TC}$ or the like. I will keep the notation $\zff$ for the stronger theory as I have no use for the weaker one in my answer.)
Now, I claim that $T$ proves $\zffm$. Axioms 1–3 are straightforward. For 4, first note that $T$ proves that every element $x$ other than the minimal element $0$ has a predecessor (as $\{y:y<x\}$ has a maximal element) and successor $S(x)$ (in particular, as mentioned in paste bee’s answer, there is no largest element, as otherwise $\{x:x\notin x\}$ would exist, leading to Russell’s paradox). Using Sets and Well-founedness, $T$ proves order induction
$$\forall x\,\bigl(\forall y<x\,\phi(y)\to\phi(x)\bigr)\to\forall x\,\phi(x),$$
which together with predecessor implies usual induction
$$\phi(0)\land\forall x\,\bigl(\phi(x)\to\phi(S(x))\bigr)\to\forall x\,\phi(x).$$
Let $\|x\|<n$ denote $\forall y\in x\,y<n$. Then given a formula $\phi(x)$, $T$ proves the formula $\psi(n)\equiv$
$$\phi(\varnothing)\land\forall x,y\,\bigl(\phi(x)\to\phi(x\cup\{y\})\bigr)\to\forall x\,\bigl(\|x\|<n\to\phi(x)\bigr)$$
by induction on $n$: if $\|x\|<0$, then $x=\varnothing$, which satisfies $\phi$ by the premise. Assuming $\psi(n)$, if $\|x\|<S(n)$, then either $\|x\|<n$ and $\phi(x)$ holds by the induction hypothesis; or $n\in x$ and $x'=x\smallsetminus\{n\}$ satisfies $\|x'\|<n$, thus $\phi(x')$ by the induction hypothesis, thus $\phi(x'\cup\{n\})$ using the premise, i.e., $\phi(x)$. Then $\forall n\,\psi(n)$ implies 4 as every $x$ satisfies $\|x\|<n$ for some $n$ using Finiteness.
$\def\N{\mathbb N}$Finally, to show that $\pa$ is not bi-interpretable with $T$, the key observation is that $T$ has lots of nonisomorphic (and not elementarily equivalent) standard models. Here, I call a model $(M,<,\in)\models T$ standard if $(M,<)$ is well founded (necessarily of order-type $\omega$). For every permutation $\sigma\colon\N\to\N$, we have $\N_\sigma=(\N,<,\in_\sigma)\models T$, where
$$n\in_\sigma x\iff\text{the $n$th bit of $\sigma(x)$ is $1$.}$$
Now, assume for contradiction that $F$ is an interpretation of $\pa$ in $T$ and $G$ is an interpretation of $T$ in $\pa$ such that $F\circ G$ is definably isomorphic to the identity self-interpretation of $T$. Fix permutations $\sigma\ne\tau$. Then $F$ induces models $\N_\sigma^F$ and $\N_\tau^F$ of $\pa$, and $G$ induces a copy of $\N_\sigma$ definable in $\N_\sigma^F$ and a copy of $\N_\tau$ definable in $\N^F_\tau$, using the same definitions. Since $\N_\sigma^F$ has full induction schema, it can define an embedding $f$ of the universe into the internal copy of $\N_\sigma$ such that $f(0)$ is the least element of $\N_\sigma$, and $f(n+1)$ is the successor (as computed in $\N_\sigma$) of $f(n)$. Since $f$ is an order embedding into a well-ordered set, its domain must be well ordered as well: that is, $\N_\sigma^F$ must be isomorphic to the standard model $\N$ of $\pa$.
The same argument applies to $\N_\tau^F$, thus in particular, $\N_\sigma^F\simeq\N_\tau^F$. But then their internal models of $T$, viz. $\N_\sigma$ and $\N_\tau$, must be isomorphic as well, as they are defined by the same formulas. This is a contradiction, as $\N_\sigma$ and $\N_\tau$ are not even elementarily equivalent (they disagree on some sentences of the form $\overline n\in\overline m$, where $\overline n$ denotes the $n$th least element according to $<$).
Note that we have used only one half of the definition of bi-interpretability, thus the argument above actually shows that $T$ is not an interpretation retract of $\pa$.
Let me add that in order to get a theory synonymous with $\pa$ (or equivalently, $\zff$), it is enough to extend $T$ with the two axioms
$$\let\eq\leftrightarrow\begin{align}
\tag1n\in x&\to n<x,\\
\tag2x<y&\eq\exists n\,\bigl(n\in y\land n\notin x\land\forall m>n\,(m\in x\eq m\in y)\bigr).
\end{align}$$
(In fact, it is sufficient to keep only one implication (either one) of the outer bi-implication in (2); the other one then follows using axioms of $T$. On the other hand, it’s quite possible some of the axioms of $T$ can be simplified in the presence of (1) and (2).)
Since $T$ proves order induction, (1) ensures that the theory includes $\in$-induction, and therefore all of $\zff$. Then (2) ensures that $x<y$ is equivalent to an inductive definition in terms of $\in$ alone (coinciding with the usual order on $\omega$ lifted by Ackermann’s bijection between $\omega$ and $V_\omega$). Thus, the resulting theory is equivalent to an extension of $\zff$ by a definition.
Note that (1) alone is not enough to make the theory bi-interpretable with $\pa$, as $\N_\sigma\models(1)$ whenever $\sigma$ satisfies $\sigma(x)<2^x$ for all $x\in\N$, thus the argument above still applies.
$\sf ZFGC$ is synonymous with the theory you defined, with an additional statement $\varphi$.
Define $x \in' y \Leftrightarrow x \in y$, except if $y$ is $\varnothing$, $\{\varnothing\}$, $\{\{\varnothing\}\}$, or a natural number; make $\varnothing$ and $\{\varnothing\}$ Quine atoms (so $\varnothing \in' \varnothing$ and $\{\varnothing\} \in' \{\varnothing\}$), $\{\{\varnothing\}\}$ into $0$ (so $x \in' \{\{\varnothing\}\} \Leftrightarrow x \in 0$), and $n$ into $n+1$ ($x \in' n \Leftrightarrow x \in n+1$). Define $C'(x) = C(x)$, except if $x$ is $\{\varnothing,\{\varnothing\}\}$, $\{\{\varnothing\}\}$ or a natural number; $C'(\{\varnothing,\{\varnothing\}\}) = \varnothing$, $C'(\{\{\varnothing\}\}) = C(0)$ and $C'(n) = C(n+1)$ (because the $\in'$ elements of $n$ are the $\in$ elements of $n+1$). Define $\varphi$ as $C(\{\varnothing,\{\varnothing\}\}) = \varnothing$.
In the other direction, define $a_1$ as $C'(\{x | x \in' x\})$, and $a_2$ as the Quine atom that is not $a_1$. Define $x \in y \Leftrightarrow x \in' y$, except if $y \in' y$ or $y$ is a natural number; if $y$ is a Quine atom, $x \in y$ iff $x = a_1$ and $y = a_2$, $x \in 0$ iff $x = a_2$, $x \in n+1$ iff $x \in' n$. Define $C(x) = C'(x)$, except if $x$ is $\{a_1,a_2\}$ or a natural number; $C(\{a_1,a_2\})$ is $a_1$ if $\varphi$ is true and $a_2$ otherwise, $C(0) = a_2$, $C(n+1) = C'(n)$.
These definitions are inverses of each other, and the axioms are all equivalent, if we're careful about the construction of $\mathbb{N}$. Define $0$ as $\{\varnothing,\{\varnothing\}\}$ (from $\in$) or $\{a_1,a_2\}$ (from $\in'$), and $S(n) = n \cup \{n\}$. None of these transitively contain $\{\{\varnothing\}\}$. We can't quite use these, because they can contain each other, but if we actually use the singletons of these numbers ($\{0\}$, $\{1\}$, $\{2\}$) as the natural numbers, it works. These numbers don't contain $\{\{\varnothing\}\}$ or each other, and also aren't $\varnothing$, $\{\varnothing\}$, $\{\{\varnothing\}\}$, or Quine atoms, so all of the elements of natural numbers are completely unchanged (except the Quine atoms). This implies that Foundation is still true, and also that which natural number a set is (or whether it is a natural number at all) will not be "unexpectedly" changed.
The theory you defined is synonymous with itself with an additional statement $\varphi$. Define $C''(x) = C'(x)$, except $C''(\{0,1\})$ is $0$ if $\varphi$ is true and $1$ otherwise, and $C''(\{n+1,n+2\}) = C'(\{n,n+1\}) + 1$. In the other direction, define $C'(x) = C''(x)$, except $C'(\{n,n+1\}) = C''(\{n+1,n+2\}) - 1$, and define $\varphi \Leftrightarrow C'(\{0,1\}) = 0$. These definitions are inverses of each other. (This paragraph could be "merged" into the previous ones, to avoid having to define a third global choice function $C''$, but I think it's probably easier this way...?)
So your theory is synonymous with $\sf ZFGC$.
Best Answer
If, for every atom $a$, $a \subseteq x$ implies $a \subseteq y$, then $x \subseteq y$. Assume $x \nsubseteq y$, then by Supplementation, there is a $z \subseteq x$ such that $\neg z \ O \ y$. By Atomicity, there is an atom $a \subseteq z \subseteq x$, and $a \nsubseteq y$ because $\neg zOy$, contradicting the assumption that if $a \subseteq x$ then $a \subseteq y$.
This implies that the usual definition $a \subseteq b \iff \forall x (x \in a \Rightarrow x \in b)$ and your definition of $\in$ from $\subseteq$ are inverses of each other, so it does not matter whether the language is $\in$ or ($\subseteq$, $\{\}$); your mereological theory is synonymous with $\sf MMK$.
MMK and MK are not synonymous. A model of MK has no nontrivial definable automorphisms, but any model of MMK has a definable automorphism by swapping the two Quine atoms. If they were synonymous, then a model of MK would also be a model of MMK (with a different $\in$), and therefore have a nontrivial definable automorphism, which is impossible. (A similar argument might work to prove they are not bi-interpretable, but I'm not sure of the details). So the issue is that we can't distinguish the two Quine atoms. (The reason this argument didn't work in the case of ZFGC was the global choice function $C$, which can be used to pick one of the Quine atoms; in MMK, while choice functions do exist, there isn't a choice of one particular one that an isomorphism has to preserve).
If we add a constant $q$ and the axiom $q \in q$ (so that we can distinguish the Quine atoms), the resulting theory is synonymous with MK, by a similar argument to my previous answer, except we use $q$ to remember which Quine atom was the empty set.