The key idea is that time crystals are externally driven at a certain frequency, but they respond at a different (in fact, slower) frequency.
First of all, terminology:
what does it have in common with the usual concept of crystals, whose whole structure can be represented by spatially replicating the unit cell. Is the time crystal an addendum to the latter by extending its spatial symmetry to also include a time symmetry?
Sort of, but it's more than that. The key property of a crystal that they're generalizing is not just that it's periodic in space, but that it's spontaneously spatially periodic. In other words, you can start with a bunch of randomly arranged atoms whose interactions are perfectly translationally invariant, and they "fall into" a crystalline lattice on their own - you don't manually need to assemble the solid by adding in one atom at a time with atomic-spacing precision.
As an extremely rough analogy, let's imagine that you tried manually "packing" atoms into a lattice by subjecting them to a spatially periodic external potential. One could imagine that if the atoms are strongly repulsive, then the external potential won't be strong enough to pack them into adjacent lattice sites, so instead they will occupy every other site in the external potential. And of course, which set of "every other site" - the even- or odd-numbered sites - is randomly and unpredictably (or "spontaneously") selected.
Similarly, you could imagine a time crystal that you're weakly driving once per second, but it's so disordered that it keeps getting "semi-stuck" in its current configuration and can't keep up with the drive, so it only flops around once every two seconds. (The atoms in the previous analogy correspond to changes in the time crystal's state, and the repulsion between the atoms in the analogy corresponds to the system not "wanting" to frequently change its state.) This may not sound incredibly exotic, but it turns out that no one's ever discovered a material that does this, and up until recently it was thought to be impossible on theoretical grounds (namely Floquet's theorem), until some very clever people thought of some loopholes. "Time crystal" is kind of a silly name, but that's what stuck.
You are correct that because time crystals inherently depend on an external drive (at least, we think they do), you can't use equilibrium stat mech and you have to adopt a slightly different definition of a "state of matter" - in this case, the property of the material response spontaneously doubling (or tripling, etc.) the period of the external drive. Also, as you say, you're unlikely to find a time crystal in nature, if by "nature" you mean "a cave in Utah." But they may not be quite as exotic as you think. For example, simply shining a classical light wave of fixed frequency can drive a solid via the AC Stark effect, so you wouldn't necessarily to do anything to fancy to get the external drive.
Edit: people usually use the word "crystal" to mean a spatially periodic structure, but that's not the sense of the word being used in the phrase "time crystal." The key point of a crystal that's being generalized is that it spontaneously breaks translational symmetry, because the Hamiltonian has a certain translational symmetry, but any ground state has less translational symmetry. For a regular spatial crystal, the individual atoms' interactions have continuous translational symmetry - if you move every atom in the same direction by an arbitrarily small amount, the energy doesn't change. But if the interactions are such that the ground state forms a crystal structure, then that crystal only has discrete translational symmetry - the crystal looks the same if you translate it by one lattice constant, but not if you translate it by a fraction of a lattice constant. The ground state still has some "residual" translational symmetry left over from the original Hamiltonian, but it's less symmetry than before, because there are fewer translation operations (by integer numbers of lattice spacings) that leave the crystal invariant than translation operations (by any amount) that leave the original Hamiltonian invariant. Mathematically, we say that the original symmetry group $\mathbb{R}$ gets spontaneously broken down to the proper subgroup $\mathbb{Z}$.
In my "repulsive atom" analogy above, the Hamiltonian has discrete translational invariance with a lattice constant set by the periodic external potential. But if the atoms repel and only fill up every second (third, etc.) site of the lattice, then the physical ground state is still spatially periodic, but with a period twice (three times, etc.) the period set by the external potential. We say that "the unit cell has spontaneously doubled," because the ground state still has some translational symmetry, but less than before (translation by an even number of original lattice spacings is still a symmetry, but translation by an odd number no longer is).
Similarly, a time crystal simultaneously doubles (or triples, etc.) the "unit cell" of time translation. If the Hamiltonian is periodic in time with period $T$, but the physical crystal's time evolution is periodic with period $2T$ ($3T$, etc.) then it "spontaneously breaks" the Hamiltonian's time translational symmetry (by any integer multiple of $T$) down to a smaller set of symmetry operations - those consisting of an integer multiple of the new period $2T$, or equivalently an even multiple of $T$. Just as the effective lattice constant doubled in our atom analogy, the "time translational unit cell" doubles from $T$ to $2T$ in our time crystal.
(Technical detail: the Hamiltonian for a time crystal is disordered in space, but perfectly periodic in time - so the time translational invariance that gets spontaneously broken is indeed a perfect symmetry of the Hamiltonian. The spatial disorder is needed in practice for rather technical reasons, but is completely unimportant conceptually.)
Interaction between marbles is very similar to the hard sphere (HS) interaction model i.e. a pair-wise potential energy which is zero if spheres do not overlap and $+\infty$ elsewhere.
Hard spheres are one of the first systems studied via computer simulation and one of the first big surprise was that by increasing pressure, they are able to crystallize from a disordered fluid to an fcc crystal, in 3D, or to a triangular lattice in 2D. After the first pioneering studies the scenario has been confirmed many times and fully understood. Moreover, in the nineties, the experiments by the Pusey's group in UK have shown that the theoretical scenario is closely followed by colloidal systems designed to mimic as closely as possible a real system of HS (Pusey, P. N., & Van Megen, W. (1986). Phase behaviour of concentrated suspensions of nearly hard colloidal spheres. Nature, 320(6060), 340.)
It is interesting to notice that the HS crystal is stable on the base of entropic reasons. Neither attraction nor quantum mechanics are needed and the density of the coexistent solid at freezing is about 30% smaller than the close packing density (which means that in the HS crystal at the freezing point spheres collides frequently but do not touch all the time). Probably one of the most interesting things about the HS solid is that is a very nice illustration why the naïf equation entropy="spacial disorder" is wrong: the HS crystal has a higher entropy per particle than the coexisting liquid.
What can be said about marbles, taking into account HS? Although their interaction is a very good representation of the HS potential, usually they lack the dynamics underlying the behavior of a true thermodynamic system. Dissipative effects are quite strong and in a short time, without an external continuous feed of energy, the kinetic energy of marbles gets dissipated. In the very old times of the study of liquids, somebody performed experiments with a 2D system of marbles in a tray put on top of a hi fi speaker as a tool to feed kinetic energy randomly. However, without such a flux of energy, what can be observed by shaking a 2D or 3D container almost filled with marbles is that, if the system is highly disordered at the beginning, after some shaking part of the "defects" are eliminated and, at least locally, the system looks like a crystalline solid at the close packing. But this is a situation not directly related with the thermodynamic transition. It has more to do with the stability with respect to perturbations of purely mechanical equilibrium configurations. As a last comment, I would add that the dynamic behavior of marble-like particles has been and still is an active research topic in the physics of granular media.
Best Answer
To put it in simple terms:
In the case of metals, surrounding the nucleus of each atom is a cloud of electrons. While some of the electrons occupy clouds that are spherically symmetric, other clouds have lobes that point in certain directions. When settling into solid crystals, those atoms strive for the lowest energy state available to them, which involves accommodating any electron orbitals that are not spherically symmetric as they settle in together.
Although this effect is subtle, it makes certain orientations of nearest-neighbor atoms more likely than others, and certain specific crystal lattices are the result.
In the case of nonmetals, the binding that occurs between the atoms relies almost completely on those asymmetric electron clouds and the result will be a crystalline structure that can be exceptionally rigid (as in the case of diamond for example).