This isn't a complete answer, but I thought it would be useful to write it out in more detail than would fit in the space provided.
First, I'd like to observe that both of the equivalent definitions of group action admit generalisations:
The definition using a homomorphism $G \to \text{Aut}(X)$ generalises straightforwardly to the case when $G$ is a (discrete) group and $X$ a vector space, giving rise to the notion of linear representations of groups.
The definition using a map $G \times X \to X$ generalises straightforwardly to the case when $G$ is a topological group and $X$ a topological space, giving rise to the notion of continuous group actions on topological spaces.
Your question seems to be asking whether we can find a definition which would encompass both branches of generalisations. It's not an unreasonable question — after all, both of the examples above converge when we have a continuous linear representation of a topological group.
Let's try to formulate categorically the equivalence of the two definitions in $\textbf{Set}$, and see how things go when we try to change to a more general category. Firstly, we note that a small group is equivalently
- a category $\mathcal{G}$ enriched over $\textbf{Set}$ with one object and all arrows invertible, and
- a set $G$ equipped with maps $m : G \times G \to G$, $e : 1 \to G$, $i : G \to G$ satisfying the group axioms.
The connection between the two definitions is expressed by the equation $\mathcal{G}(*, *) = G$, where $*$ is the unique object in $\mathcal{G}$. Using the first definition, a group action of $G$ is simply any functor $\mathscr{F} : \mathcal{G} \to \textbf{Set}$. Focusing on the hom-sets, we see that we have a monoid homomorphism $G \to \textbf{Set}(X, X)$, where $X = \mathscr{F}(*)$, and since the domain is a group, the codomain must lie in $\text{Aut}(X) \subseteq \textbf{Set}(X, X)$. Thus we have the first definition of group action.
Now, we recall that $\textbf{Set}$ is a cartesian closed category (indeed, a topos), so in particular we have the exponential objects $Y^X$, which are defined by following the universal property: $\text{Hom}(Z \times X, Y) \cong \text{Hom}(Z, Y^X)$ naturally in $Z$ and $Y$. Thus, we may identify the map $G \to \textbf{Set}(X, X)$ with a map $G \times X \to X$, and translating the homomorphism axioms through this identification gives the second definition of group action.
Consider a group object $G$ in a cartesian monoidal category $(\mathcal{C}, \times, 1)$. This may be viewed as a category $\mathcal{G}$ enriched over $\mathcal{C}$. Then, an action of $G$ on an object $X$ in another category $\mathcal{D}$ enriched over $\mathcal{C}$ is simply a $\mathcal{C}$-enriched functor $\mathcal{G} \to \mathcal{D}$. If $\mathcal{D}$ is such that there is a $\mathcal{C}$-enriched "forgetful" functor $U : \mathcal{D} \to \mathcal{C}$ and $\mathcal{C}$ is a cartesian closed category, we may do the same trick as before and obtain an arrow $G \times X \to X$ in $\mathcal{C}$.
In particular, if $\mathcal{C}$ is a cartesian closed category, it is enriched over itself. Indeed, consider the counit $\epsilon_{Z,X} : Z^X \times X \to Z$ of the product-exponential adjunction. If we compose with $\text{id} \times \epsilon_{X,Y} : Z^X \times X^Y \times Y \to Z^X \times X$, we get a map $Z^X \times X^Y \times Y \to Z$, which we may take transpose to obtain a map $Z^X \times X^Y \to Z^Y$, which is the composition of arrows. Thus we may recover the notion of continuous group actions at least in the case where both the group and the space being acted on are compactly-generated Hausdorff spaces...
A universal cover of a connected groupoid $B$ is a discrete fibration $E \to B$ where $E$ is a simply connected groupoid.
As it turns out, there is a very easy construction.
Choose a point $b$ of $B$ and take $E$ to be the slice $B_{/ b}$.
$E$ has a terminal object, so it is contractible, hence simply connected a fortiori.
The projection $E \to B$ is a fibration and it is easy to check that the fibres are discrete.
Hence we have the desired universal cover of $B$.
Incidentally, if you do this with $\infty$-groupoids instead of 1-groupoids you end up with a (not necessarily discrete) fibration $E \to B$ where $E$ is contractible, so in some sense I have cheated and exploited the 1-dimensionality of $B$ in the above.
Best Answer
At a certain level, an $\infty$-group $G$ is not so different from the groups you are used to. $G$ has an underlying space, analogous to the underlying set of a group, a unit $e:*\to G$, a multiplication $m:G\times G\to G$, and an inversion map $i:G\to G$. Where things begin to get interesting is the sense in which these operations satisfy the group axioms. It is no longer true that $m(m(g_1,g_2),g_3)=m(g_1,m(g_2,g_3))$; rather, this associativity axiom becomes yet another piece of structure, an associator map $a:G\times G\times G\times I\to G$ giving a homotopy between the two sides of the above equation. Similarly, we have to introduce "unitors" replacing the equations $m(g,e)=g$ and $m(e,g)=g$ with homotopies, and similarly for the equations involving the inverse.
Furthermore, this isn't all! In fact a group satisfies many more equations than those in the usual axiomatization. For instance, there are five different ways to parentheses four letters: $g_1(g_2(g_3g_4)),(g_1g_2)(g_3g_4),((g_1g_2)g_3)g_4),(g_1(g_2g_3))g_4,$ and $g_1((g_2g_3)g_4)$. In a group, these are all equal, and this follows from the associativity axiom. In an $\infty$-group, we have homotopies between these five parenthesizations, interpreted as maps $G^4\to G$. In fact, we can paste these homotopies together into a map $G^4\times \partial P\to G$, where $P$ is a regular pentagon in the plane. We would like to know that there is, in essence, only one way to associate two products into each other-it would be a bad generalization of group theory if we could follow a nontrivial loop in $G$ by simply associating one word back to itself in some complicated sequence! Thus part of the structure of an $\infty$-group is an extension of the above map to the pentagonator $\pi:G^4\times P\to G$.
And we're not done yet. In fact, there are infinitely many levels of structure needed for describing all the ways of associating longer and longer words alone, and the spaces which are $I$ for the associator and $P$ for the pentagonator continue growing in dimension and combinatorial complexity. Stasheff gave the first complete description of this part of the structure of an $\infty$-group, which he called an $A_\infty$-space, for a space with a multiplication which is Associative up to a homotopy which is itself well defined up to a homotopy which is well defined up to...Stasheff's original papers are still excellent reading on this topic.
The nLab's "group object" in an $\infty$-category is closely related to Stasheff's notion of $A_\infty$-space and in similar ways to the elementary notion of a group. The "simplical object in an $\infty$-category" that is the underlying structure of a group object is supposed to represent the group $G$ together with all its finite powers $G^n$ (including $n=0$) while the simplicial face maps correspond to the canonical projections between $G^n$ and $G^m$, the degeneracies correspond to various ways of mapping $G^m$ to $G^n$ by inserting copies of the unit, and the various pullback squares cleverly encode the multiplication, inverse, and all the infinite tower of homotopies witnessing the axioms as in the previous paragraph. This is also closely connected to Lawvere's perspective on groups: a model of the Lawvere theory of groups in a category $C$ is exactly all the stuff I just said, except that the homotopies are allowed again to be equations. So it's no different than an ordinary group object, except insofar as we don't pick out particular operations and axioms as privileged.
This is a pretty complicated structure! A large amount of work in the last fifty years of algebraic topology has been on how best to understand these objects. One fundamental theorem is that an $A_\infty$-space has a delooping if and only if it is actually an $\infty$-group: having a delooping means it is homotopy equivalent to the space of based loops in some pointed connected space, which is itself unique up to homotopy equivalence. And a map of $\infty$-groups, i.e. some kind of homomorphism appropriately preserving all the huge mess of structure up to a huge number of homotopies, is nothing more than a map of their deloopings. (This statement is a little bit cleaner than the reality, but it's close.) This is the equivalence between $\infty$-groups and pointed connected objects you mention. This has less of an obvious analogue to ordinary group theory, but it's still there: it's simply the well-known perverse definition of a group as a groupoid with a single object. The reason the equivalence is so much more interesting in $\infty$-category theory is that pointed connected $\infty$-groupoids, i.e. pointed connected spaces, are usually not defined with algebraic operations of composition and inversion of loops, so much less structure has to be carried around in defining them and in particular their maps. Another simplification is that every $\infty$-group is appropriately equivalent to the geometric realization of a simplicial group, that is, a completely ordinary group object in the ordinary category of simplicial sets. At least the equivalence between pointed connected spaces and simplicial groups is the oldest of all these results-it goes back to Kan in the early '60s.
Anyway, hopefully that's given a bit more of an idea of what's going on. There are many approaches to the concept, largely because all of the approaches become intolerably complicated in one way or another. This situation is characteristic of $\infty$-category theory, and in the current state of knowledge it appears unavoidable that appending "$\infty$" to a familiar object creates substantial complications.