Sorry for the long post but this is a personal piece of maths, and I needed to be more precise as possible.
There exists a well known equivalence between the category of $G$-sets and the category of functors $Fun(G,\mathbf{Sets})$ (viewing $G$ as a category with a single object $*$): given a set $X$ just consider the unique functor sending $*$ into $X$; functoriality determines the well known classical permutation representation $G\to Sym(X)$, which easily leads to the equivalent notion of an action as a map $G\times X\to X$ with suitable properties.
It is also well known that the notion of "action" of a "group object" can be stated in any category with finite products and a terminal object (say, $1$).
I would like to relate the two notions: I started noticing that the first relation lacks of elasticity, being formulated in the particular case of the category of sets. My first task is hence to generalize the notion of action of a group on some $X$, object in $\mathbf C$ (category w. finite products and a terminal object). It seems to me that I can define it as a functor $G\to \mathbf C$ sending $*$ into $X$: Functoriality allow me to think that any $g\in G$, seen as an isomorphism $g\colon *\to *$, corresponds via $F$ to an isomorphism in $\mathbf C$, then $G$ is related to a subset(subgroup) in $\hom_\mathbf C(X,X)$.
So, it seems we recovered the notion of permutation representation in this more general context. Futhermore, it seems to me that when we consider, say, a functor $F\colon G\to \mathbf{Top}$ ($G$ a group in that category, i.e. a topological group), mere functoriality allows me to say that $G\times X\to X$ is a continouos map (similarly consider then the subcategory of $\mathbf{Top}$ made by differentiable manifolds, $G$ is then a Lie group and the action a smooth map).
Can we go any further? Moerdijk defines the action of a groupoid on a space $\mathbf G=(s,t:G_\text{mor} \to G_\text{ob})$ as an arrow $\mu\colon G_\text{mor}\times_{G_\text{ob}} E\to E$ with suitable properties. This is partly similar to an idea I got yesterday, thinking to a suitable way to expand the notion of $G$ acting on something.
In a few words, I start identifying $G$ and $\hom(*,*)$, groups in set-theoretic sense wrt the composition. Say that an action of $G$ on an object $X$ in $\mathbf C$ is a function $\hom(*,*)\times \hom(1,X)\to \hom(1,X)$ which is an action in the classical set-theoretic sense: is it formally correct? Can this approach be applied in a "real case"? Can you provide me a reference for the definition I gave of Moerdijk groupoid-action, which I read dunno-where on MO?
Thanks everybody.
Best Answer
This isn't a complete answer, but I thought it would be useful to write it out in more detail than would fit in the space provided.
First, I'd like to observe that both of the equivalent definitions of group action admit generalisations:
The definition using a homomorphism $G \to \text{Aut}(X)$ generalises straightforwardly to the case when $G$ is a (discrete) group and $X$ a vector space, giving rise to the notion of linear representations of groups.
The definition using a map $G \times X \to X$ generalises straightforwardly to the case when $G$ is a topological group and $X$ a topological space, giving rise to the notion of continuous group actions on topological spaces.
Your question seems to be asking whether we can find a definition which would encompass both branches of generalisations. It's not an unreasonable question — after all, both of the examples above converge when we have a continuous linear representation of a topological group.
Let's try to formulate categorically the equivalence of the two definitions in $\textbf{Set}$, and see how things go when we try to change to a more general category. Firstly, we note that a small group is equivalently
The connection between the two definitions is expressed by the equation $\mathcal{G}(*, *) = G$, where $*$ is the unique object in $\mathcal{G}$. Using the first definition, a group action of $G$ is simply any functor $\mathscr{F} : \mathcal{G} \to \textbf{Set}$. Focusing on the hom-sets, we see that we have a monoid homomorphism $G \to \textbf{Set}(X, X)$, where $X = \mathscr{F}(*)$, and since the domain is a group, the codomain must lie in $\text{Aut}(X) \subseteq \textbf{Set}(X, X)$. Thus we have the first definition of group action.
Now, we recall that $\textbf{Set}$ is a cartesian closed category (indeed, a topos), so in particular we have the exponential objects $Y^X$, which are defined by following the universal property: $\text{Hom}(Z \times X, Y) \cong \text{Hom}(Z, Y^X)$ naturally in $Z$ and $Y$. Thus, we may identify the map $G \to \textbf{Set}(X, X)$ with a map $G \times X \to X$, and translating the homomorphism axioms through this identification gives the second definition of group action.
Consider a group object $G$ in a cartesian monoidal category $(\mathcal{C}, \times, 1)$. This may be viewed as a category $\mathcal{G}$ enriched over $\mathcal{C}$. Then, an action of $G$ on an object $X$ in another category $\mathcal{D}$ enriched over $\mathcal{C}$ is simply a $\mathcal{C}$-enriched functor $\mathcal{G} \to \mathcal{D}$. If $\mathcal{D}$ is such that there is a $\mathcal{C}$-enriched "forgetful" functor $U : \mathcal{D} \to \mathcal{C}$ and $\mathcal{C}$ is a cartesian closed category, we may do the same trick as before and obtain an arrow $G \times X \to X$ in $\mathcal{C}$.
In particular, if $\mathcal{C}$ is a cartesian closed category, it is enriched over itself. Indeed, consider the counit $\epsilon_{Z,X} : Z^X \times X \to Z$ of the product-exponential adjunction. If we compose with $\text{id} \times \epsilon_{X,Y} : Z^X \times X^Y \times Y \to Z^X \times X$, we get a map $Z^X \times X^Y \times Y \to Z$, which we may take transpose to obtain a map $Z^X \times X^Y \to Z^Y$, which is the composition of arrows. Thus we may recover the notion of continuous group actions at least in the case where both the group and the space being acted on are compactly-generated Hausdorff spaces...