I've managed to answer a few of the latter questions, and please look below for a solution in the finite-degree case.
Theorem. There is an uncountable graph $\Gamma$ that is not a Cayley graph in the set-theoretic universe $V$, but which is a Cayley graph in a larger set-theoretic universe, obtained by forcing.
Proof. Let $\Gamma$ be a directed graph that is a tree, such that all vertices have infinite in-out degree, but such that some of these degrees are countable and some uncountable. For example, perhaps all the in-degrees are countable and all out-degrees are uncountable. It is not difficult to construct such a graph. Clearly, $\Gamma$ cannot be a Cayley graph, since the degrees don't match. Denote the (current) set-theoretic universe by $V$, and perform forcing to a forcing extension $V[G]$ in which $\Gamma$ is countable. (It is a remarkable fact about forcing that any set at all can become countable in a forcing extension.) In the extension $V[G]$, the graph $\Gamma$ is the countable tree in which every vertex has countably infinite in-out degree. Thus, in the forcing extension, $\Gamma$ is the Cayley graph of the free group on countably many generators. $_\square$
Theorem. There is a graph $\Gamma$ which is not a Cayley graph, but every finite subgraph of $\Gamma$ is part of a Cayley graph. Indeed, every countable subgraph of $\Gamma$ is part of a Cayley graph. Every countable subgraph of $\Gamma$ can be extended to a larger countable subgraph of $\Gamma$ that is a Cayley graph.
Proof. The same graph $\Gamma$ as above works. Every countable subgraph of $\Gamma$ involves only countably many edges, and can be placed into a countable subgraph of $\Gamma$ that is the Cayley graph of the free group on countably many generators. $_\square$
The conclusion is that you cannot tell if an uncountable graph is a Cayley graph by looking only at the finite subgraphs, or indeed, by looking only at its countable subgraphs.
These observations show that in the uncountable case, one should just restrict the question at the outset to graphs satisfying the degree-matching condition.
Let me also give the fuller details for the finite-degree case, following the suggestion of François Dorais (and my tree-of-attempts argument).
Theorem. There is a finitistic condition on
countable graphs $\Gamma$ that holds exactly of the
finite-degree Cayley graphs. Specifically, the set of
finite-degree countable Cayley graphs has complexity at
most $\sum_4^0$ in the arithmetic hierarchy.
Proof. Let us suppose that the graph $\Gamma$ is given to us
simply as a binary relation on $\omega$, that is, an element
of $2^{\omega\times\omega}$. The assertion that the
graph is connected has complexity
$\prod_2^0$, since you must only say that
every two vertices are connected by a finite path. The
assertion that the graph has finite degree and all in-out
degrees are the same has complexity
$\sum_4^0$, since you can say "there
is a natural number $k$ such that for all vertices $v$ there
are $k$ vertices $w_1,\dotsc,w_k$ pointing at
$v$, such that no other vertex points at $v$ (and similarly for
pointing out).
Now, suppose that $\Gamma$ is a connected directed graph and
all in-out degrees are $k$. Fix a node $e$ that will represent
the group identity (if $\Gamma$ is a Cayley graph, any node
will do since they all look the same, so take $e=0$). Let us
refer to the $k$ nodes pointed at from $e$ as the set of
generators (since if $\Gamma$ really is a Cayley graph,
these will be the generators). Define that $p$ is a partial
labeling of the edges of $\Gamma$ with generators, if $p$ is
a function defined in a finite distance ball of $e$ in
$\Gamma$, where $p$ labels each arrow in that ball with a
generator. Such a labeling is coherent if first, for
every node every generator is used once going out from that
node and once going into that node, and second, if for
every node $v$, if there is a loop starting and ending at $v$,
then the word $w$ obtained from that loop works as a loop
from every node to itself. We only enforce the requirements on coherence of $p$ as far as $p$ is defined (since it is merely a partial function). Thus, a coherent labeling is an attempt to
make a labeling of the the graph into a Cayley graph, that
has not run into trouble yet.
Let $T$ be the collection of finite coherent labelings,
labeling all edges on the first $n$ nodes for some $n$. This is
a finitely branching tree under inclusion.
I claim that $\Gamma$ is a Cayley graph if and only if there
are arbitrarily such large coherent labelings. The forward
direction is clear, since we may restrict a full coherent
labeling to any finite subgraph. Conversely, if we can find
a coherent labeling for any finite subgraph of $\Gamma$,
then the tree $T$ is infinite and finitely branching. Thus,
by Konig's lemma there is an infinite branch. Such a branch
labels all the edges in $\Gamma$ and satisfies the coherence
condition. Such a labeling exactly provides a group
presentation with Cayley graph $\Gamma$.
The complexity of saying that every initial segment of the
nodes admits a coherent labeling is
$\prod_2^0$, since you must say for every
$n$, there is a labeling $p$ of the first $n$ nodes such that $p$
is coherent. Being coherent is a
$\Delta_0^0$ requirement on $p$. $_\square$
In particular, to recognize when a graph is a finitely
generated Cayley graph does not require one to quantify
over infinite objects, and so for finite degree graphs,
this is an answer to the main question.
I was surprised that the part of the description driving
the complexity is the assertion that the degrees must all
match, rather than the assertion that there are coherent
labelings on the finite subgraphs. Can this be simplified?
This still leaves the case of countably many generators
wide open. Also, this still doesn't answer the question
about having a computable graph $\Gamma$, which is a Cayley
graph, but which has no computable presentation. Such an
example would be very interesting. Even in the finite
degree case, as I mentioned in the question, the best the
argument above produces is a low branch.
If Kazhdan's property (T) is reflected in the structure of the Cayley graph, then not in a very geometric way.
Steve Gersten (that is what I read in the book by B. Bekka, P. de la Harpe and A. Valette) was the first who found that Kazhdan's property (T) is not invariant under quasi-isometry. The reason is not complicated. If a central extension
$$1 \to \mathbb Z \to \Gamma \to \Lambda $$
is obtained from a bounded cocycle $c \colon \Lambda \times \Lambda \to \mathbb Z$, then $\Gamma$ is quasi-isometric to $\Lambda \times \mathbb Z$. This situation arises for $\Lambda$ a cocompact lattice in a simple real Lie group with infinite fundamental group; such as $SU(2,2)$. Then, $\Gamma$ is the inverse image of $\Lambda$ in the universal covering. In this situation, one actually obtains (for suitable generating sets) a bi-Lipschitz equivalence of Cayley graphs.
Now, $\Lambda \times \mathbb Z$ does not have Kazhdan's property (T) since it surjects onto $\mathbb Z$, but $\Gamma$ has Kazhdan's property (T) inheriting it from the universal cover of $SU(2,2)$.
Best Answer
This paper, by A. Valette, is a survey devoted to this question, although he's more interested in infinite groups. In the infinite case, the "adjacency matrix" is a bounded operator on $\ell^2(\Gamma)$, and its spectrum makes sense. Of course, it depends on the generating set.
One of the first results he mentions is a theorem of Kesten : it is possible to recover the fact that $G$ is amenable, or free, by looking at this spectrum.