Simplicial sets and simplicial complexes lie at two ends of a spectrum, with Delta complexes, which were invented by Eilenberg and Zilber under the name "semi-simplicial complexes", lying somewhere in between. Simplicial sets are much more general than simplicial complexes and have the great advantage of allowing quotients and products to be formed without the necessity of subdivision, as is required for simplicial complexes. In this way simplicial sets are like CW complexes, only more combinatorial or categorical. The price to pay for this is that simplicial sets are perhaps less geometric, or at least not as nicely geometric as simplicial complexes. So the choice of which to use may depend in part on how geometric the context is. In some areas simplicial sets are far more natural and useful than simplicial complexes, in others the reverse is true. If one drew a Venn diagram of the people using one or the other structure, the intersection might be very small.
Delta complexes, being something of a compromise, have some of the advantages and disadvantages of each of the other two types of structure. When I wrote my algebraic topology book I had the feeling that Delta complexes had been largely forgotten over the years, so I wanted to re-publicize them, both as a pedagogical tool in introductory algebraic topology courses and as a sort of structure that arises very naturally in many contexts. For example the classifying space of a category is a Delta complex.
Incidentally, I've added 5 pages at the end of the Appendix in the online version of my book going into a little more detail about these various types of simplicial structures. (I owe a debt of thanks to Greg Kuperberg for explaining some of this stuff to me a couple years ago.)
(A note: I am going to regard simplicial sets as also defined on the empty ordinal as well, with $X(\emptyset) = *$, which is required for the join formula. This is implicit in your first definition and will remove the need for two extra cases for $d_i$ at the end.)
Regarding the "minor" question. The short explanation is that this follows by decomposing the Hom-set according to the preimage of $c$ and $c'$ in $n$, and observing that each decomposition of $n$ provides an initial choice.
In more category-theoretic language, one way to rewrite the convolution is using the "over" category:
$$
(X \star S)_n = \int^{[n] \to [c] \boxplus [c']} X_c \times S_{c'}
$$
where now the coend is taken over the comma category $n \downarrow \boxplus$ whose objects are triples $([c],[c'],f)$ of a pair of objects of $\Delta$ and a morphism from $[n]$ to their ordinal sum. We note that this comma category decomposes as a disjoint union of categories: each $([c],[c'],f)$ determines a decomposition $[n] = f^{-1} [c] \cup f^{-1} [c']$ into a disjoint union, and morphisms preserve such a decomposition. Therefore,
$$
[n] \downarrow \boxplus \simeq \coprod_{[n] = I \cup I'} (I \downarrow \Delta) \times (I' \downarrow \Delta)
$$
This makes the coend decompose:
$$
(X \star S)_n = \coprod_{[n] = I \cup I'} \int^{I \to [c], I' \to [c']} X_c \times S_{c'}
$$
However, the comma category $(I \downarrow \Delta)\times(I' \downarrow \Delta)$ has an initial object: $I \times I'$ itself. Thus, the coend degenerates down to simply being the value:
$$
(X \star S)_n = \coprod_{[n] = I \cup I'} X_{|I|} \times S_{|I'|}
$$
This is slightly different notation for the second definition of the join that you gave.
Now, as for the boundary formulas.
The definition of $d_i$ is as follows. For each $0 \leq i \leq n$, there is a unique map $d^i:[n-1] \to [n]$ in $\Delta$ whose image is $[n] \setminus \{i\}$: $d^i(x) = x$ for $x < i$, and $d^i(x) = x+1$ for $x \geq i$. The induced map $(X \star S)_n \to (X \star S)_{n-1}$ is the map induced by applying the contravariant functor to $d^i$.
Since $(X \star S)_n$ is a disjoint union of sets, it suffices to show that the formula is correct on $X(I) \times S(I')$ for all decompositions of $[n]$ into $I \cup I'$, where $|I| = j+1$ and $|I'| = k+1$. There are two possibilities: either $i \in I$ when $0 \leq i \leq j$, or $i \in I'$ when $j < i \leq n$.
In either case, the map $[n-1] \to [n] = I \cup I'$ induces, by taking preimages, a unique ordered decomposition $[n-1] = J \cup J'$ of $[n-1]$. If $i \in I$, then $J$ has size $|I| - 1$ and $J'$ is mapped isomorphically to $I'$ by $d^i$. In this case, the map $d^i$ is isomorphic to the map $d^i \boxplus id$ on $[j-1] \boxplus [k] \to [j] \boxplus [k]$. If $i \in I'$, we have the reverse possibility, with $d^i$ isomorphic to $id \boxplus d^{i-j-1}$ (the upper index necessary because inserting the identity at the beginning adds $j+1$ elements to the ordered set at the beginning).
In the case $0 \leq i \leq j$, the induced map
$$
d_i: X(I) \times S(I') \to \coprod_{[n-1] = K \cup K'} X(K) \times S(K')
$$
is therefore the map $X(d^i) \times id: X(I) \times S(I') \to X(J) \times S(J')$, followed by the inclusion into the coproduct. In the case $j < i \leq n$, the map is $id \times S(d^{i-n-1})$ followed by inclusion.
This recovers the formula for $d_i$ that you have written down, up to inserting copies of a point $*$ as in the remark at the beginning.
Best Answer
Well, the deleted join has been studied. Somewhat informally, that's where you take the join of a simplicial complex $\Delta$ with itself, then delete the faces $\sigma_1 \cup \sigma_2$ in $\Delta * \Delta$ such that $\sigma_1 \cap \sigma_2 \neq \emptyset$ in $\Delta$. The deleted join is nice from the Borsuk-Ulam point of view, because it admits a free $\mathbb{Z}_2$ action by exchanging the two copies of $\Delta$.
A related construction is the Bier sphere, where you replace the second copy of $\Delta$ with the combinatorial Alexander dual.
Matousek discusses both deleted joins and Bier spheres in his book Using the Borsuk-Ulam Theorem.