In terms of an intuitive understanding of categories, I've been told to think of a category as "objects with some structure" and a morphism as "a transformation preserving the structure". (This understanding obviously falls apart when considering abstract categories, but seems to me to be unrelated to my problem, which is with its interpretation on concrete categories)
An isomorphism is then meant to indicate that the two objects in question have exactly identical structure. It seems to me that we have no guarantee that this notion is captured in general by our general definition of isomorphism.
If you asked someone who only knew of algebraic structures what an isomorphism on concrete categories ought to be, the answer would obviously be "a bijective morphism", but this fails for topological spaces.
By the same token, while it's certainly true that "having a morphism with inverse" ensures that two objects have identical structure for all the concrete categories I can think of, how do we know there isn't some kind of structure that violates our current notion of isomorphism in the same way that topological spaces violate the "naive" algebraic notion?
Best Answer
A standard way of trying to explain how an isomorphism between two objects of any category means precisely that they look exactly the same as far as that category is concerned is via the Yoneda embedding.
In our situation we should observe that two objects $x,y$ in a category $C$ are isomorphic if and only if they represent the same functor, that is, if and only if there is a natural isomorphism between the functors $Hom_C(-,x)$ and $Hom_C(-,y)$. In other words, $x$ and $y$ are isomorphic if and only if they admit isomorphic sets of maps from all other objects of $C$. Since a natural isomorphism is merely a natural transformation whose components are bijections of sets, this actually reduces the general notion of isomorphism to the algebraic concept of "bijective morphism", showing that in this sense every category is a category of algebraic structures. In particular, this is not circular-we don't already need to know what an isomorphism is in general to know what a natural isomorphism is!
So, if we define "$x$ and $y$ have precisely the same structure" as "$x$ and $y$ look the same as codomains to all other objects of $C$", formally, $x$ and $y$ represent naturally isomorphic functors, then the conservativity of the Yoneda embedding proves that our notion of isomorphism is the correct one.
One possible complaint is that you might wonder why we shouldn't say that $x$ and $y$ are "the same" if they admit the same maps to every other object of $C$, rather than from. Luckily, these are equivalent, since objects are isomorphic in $C$ if and only if they are isomorphic in $C^\mathrm{op}$. The main remaining complaint is that we might somehow want more of $x$ and $y$ than that the other objects of $C$ see them as being the same. But in that case, it seems clear to me that we're no longer talking about any notion of sameness in $C$.