There is an important difference, relevant to the original question, between the two kinds of $p$-adic integrals mentioned by Kevin in his comments. Because I see frequent confusion on this issue, I thought I'd comment.
The 'usual' $p$-adic integrals as you might see in, say, Tate's thesis on L-functions or the adelic theory of automorphic forms, are volume integrals, with respect to a measure, typically on some group. This kind of volume integral can also be easily defined on arbitary varieties, and you will see plenty in Weil's book on Tamagawa numbers, or in papers on motivic integration. Coleman integration, on the other hand, is a $p$-adic analogue of line integrals, and comes up most naturally in discussing the holonomy of vector bundles with connection on a variety over a $p$-adic field (often interpreted as isocrytals). These, therefore, should be the right quantities to relate to a Cauchy formula. However, unfortunately (and fortunately), it doesn't work. The reason is that Coleman integration is a line integral along a canonical path between two points on a variety over the $p$-adics. So there is a canonical holonomy in the theory, at least if you just want to compute it for a bundle with unipotent connection, that is, one that has a strictly upper-triangular connection form. This is where a mysterious 'crystalline' structure on the space of paths is used, whereby there is a unique path invariant under the action of the Frobenius. The notion of a path, by the way, uses the Tannakian formalism in this context. For a very quick overview of this approach, you can look at section 2 of this paper: http://www.ucl.ac.uk/~ucahmki/siegelinv.pdf
Breuil's paper linked from Chandan's answer should provide a more systematic overview.
Anyways, because of the canonical paths in Coleman's theory, there can be no holonomy around a loop, and hence, no Cauchy formula. I was told quite a few years ago by Berkovich that he has a theory of line integrals on Berkovich spaces that are path dependent in interesting ways, but I've never looked into it.
Added: I realize I didn't mention above the connection between holonomy and usual integration of a one-form $A$. You get this by considering the connection
$$d+\begin{bmatrix}0& A; \\
0& 0\end{bmatrix}$$
on the trivial bundle of rank two. One view of Coleman integration is that the holonomy $H_a^b$ from $a$ to $b$ is defined first. And then, the naive integral is defined by the fomula
$$H_a^b=\begin{bmatrix}1& \int_a^bA ;\\
0& 1\end{bmatrix}$$
If you use this definition, then $\zeta_p(k)$ is zero at negative even integers $k$, so by a $p$-adic continuity argument, it must also be zero at positive even integers.
What about the odd integers? At $k = 1$ there is a pole, unsurprisingly. At odd $k \ge 3$ the value is extremely mysterious, just as the complex zeta values $\zeta(k)$ are. There is an interpretation of the odd $p$-adic zeta values in terms of a $p$-adic regulator map in $K$-theory (see this question), but this is tough to get explicit information out of.
As an example of how little we understand these numbers, I believe it's an open problem whether the values $\zeta_p(k)$ for odd $k \ge 3$ are always non-zero, although this is certainly expected.
Best Answer
There are the lecture notes by Serre (Springer lecture notes 1500) of a course on Lie groups and Lie algebras. He proves the implicit function theorem for analytic functions on a $p$-adic manifold (not smooth functions, though) The following is a link:
http://www.amazon.com/Lie-Algebras-Groups-University-Mathematics/dp/3540550089/ref=cm_cr_pr_product_top