While it is true that algebraic structures based on binary operations are very common, other structures exist, are being studied, and are very important. Examples include:
$A_\infty $-spaces, where homotopy considerations mandate not just one binary operation, but an $n$-ary operation for all $n\in \mathbb N$.
Malcev operations, are examples of structures based on ternary operations.
operads also have $n$-ary operations.
For many of the structures above things like modules and actions make sense, and themselves involve $n$-ary operations.
So, mathematicians certainly do study such structures. Perhaps the reason they are not commonly introduced at the undergraduate level is that these structures are more complicated than ones based on binary operations.
As for the comment made by your professor, I can think of many ways elements can be combined to give a new element, so I really don't know what is meant by that.
And, since you are wondering if such $n$-ary based algebraic structures are too complicated for undergraduates, I'll just mention that of the three I mention above, $A_\infty $ - space are quite complicated but Malcev operations and operads are not. Operads come in many flavours, and if one considers what are known as coloured planar non-enriched operads (probably the simplest kind of operad) then this is a structure that can be understood by a first year student (and this is actually quite an important class of operads, so not just a toy algebraic structure). The reason why these structures are not introduced early on has more to do with the fact that university curricula and textbooks change and adapt very slowly. They very rarely reflect current trends. In 100 years it is likely that operads will make it to first year or second year textbooks, much like groups do today.
And while on the subject, one must also consider algebraic structures with operations of infinite arity. These too exist and provide some surprising examples. For instance, it is classical result that the category of compact Hausdorff spaces is algebraic, and that means that that category can be thought of as consisting of algebraic structures with operations of arity $\infty $. Other important examples include complete lattices.
Well, it usually goes the other way. A property $P$ of a topological space $X$ is deserved to be called topological if $P(X)$ holds if and only if $P(Y)$ holds whenever $X$ and $Y$ are homeomorphic. An example of a topological property is "$X$ is connected" while an example of a non-topological property is "$X$ is a subset of $\mathbb{R}^n$. The latter can be upgraded to a topological property by requiring instead "$X$ can be embedded in $\mathbb{R}^n$".
With this convention, given a topological space $X$ there is a smart-ass topological property one can define using $X$: "$P(Z)$ holds iff $Z$ is homeomorphic to $X$". Since homeomorphism is an equivalence relation, this is indeed a topological property and clearly $X$ satisfies $P$. If $Y$ is another space that satisfies the same topological properties as $X$, then $P(Y)$ must hold and so $X$ and $Y$ are homeomorphic.
Best Answer
Yes, there are such structures, e.g. a cohomology ring:
https://en.wikipedia.org/wiki/Cohomology_ring
Actually they are quite important. For example by analysing the cohomology ring of spheres it can be deduced that there is no topological group structure on spheres except for dimensions $0, 1, 3$ and somewhat weaker (i.e. not associative) topological group structure in dimension $7$.
Actually in case of topological groups this goes even further. The cohomology ring (of a topological group) becomes a Hopf algebra which is a very rich algebraic structure: it's a vector space with multiplication and comultiplication. So it has at least four operators ($+$, scalar multiplication, vector multiplication, comultiplication).