I'd like to write a better response, but I must be brief.
For now, let me offer some places to read. Long story short, it is predicted that there's a relationship between special values of $p$-adic $L$-functions and syntomic regulators (which are the analogue of Beilinson's regulators in the $p$-adic world).
The beautiful paper of Manfred Kolster and Thong Nguyen Quong Do is, I think, a very readable resource.
The best results I know in this direction are Besser's papers here and here, which use rigid syntomic cohomology.
Besser's overview talk at the conference in Loen (notes available here) was a real joy.
There is an important difference, relevant to the original question, between the two kinds of $p$-adic integrals mentioned by Kevin in his comments. Because I see frequent confusion on this issue, I thought I'd comment.
The 'usual' $p$-adic integrals as you might see in, say, Tate's thesis on L-functions or the adelic theory of automorphic forms, are volume integrals, with respect to a measure, typically on some group. This kind of volume integral can also be easily defined on arbitary varieties, and you will see plenty in Weil's book on Tamagawa numbers, or in papers on motivic integration. Coleman integration, on the other hand, is a $p$-adic analogue of line integrals, and comes up most naturally in discussing the holonomy of vector bundles with connection on a variety over a $p$-adic field (often interpreted as isocrytals). These, therefore, should be the right quantities to relate to a Cauchy formula. However, unfortunately (and fortunately), it doesn't work. The reason is that Coleman integration is a line integral along a canonical path between two points on a variety over the $p$-adics. So there is a canonical holonomy in the theory, at least if you just want to compute it for a bundle with unipotent connection, that is, one that has a strictly upper-triangular connection form. This is where a mysterious 'crystalline' structure on the space of paths is used, whereby there is a unique path invariant under the action of the Frobenius. The notion of a path, by the way, uses the Tannakian formalism in this context. For a very quick overview of this approach, you can look at section 2 of this paper: http://www.ucl.ac.uk/~ucahmki/siegelinv.pdf
Breuil's paper linked from Chandan's answer should provide a more systematic overview.
Anyways, because of the canonical paths in Coleman's theory, there can be no holonomy around a loop, and hence, no Cauchy formula. I was told quite a few years ago by Berkovich that he has a theory of line integrals on Berkovich spaces that are path dependent in interesting ways, but I've never looked into it.
Added: I realize I didn't mention above the connection between holonomy and usual integration of a one-form $A$. You get this by considering the connection
$$d+\begin{bmatrix}0& A; \\
0& 0\end{bmatrix}$$
on the trivial bundle of rank two. One view of Coleman integration is that the holonomy $H_a^b$ from $a$ to $b$ is defined first. And then, the naive integral is defined by the fomula
$$H_a^b=\begin{bmatrix}1& \int_a^bA ;\\
0& 1\end{bmatrix}$$
Best Answer
Introduce ${\mathbf Z}_p$ as "formal" infinite base $p$ expansions where you add and multiply by carrying (any other description will probably take too long and not be concrete). Show them the series for $-1$ in ${\mathbf Z}_3$ is $2 + 2\cdot 3 + 2 \cdot 3^2 + 2 \cdot 3^3 + \cdots$ by adding 1 to that, carrying, and killing off a new term at each step so the sum is 0. Then emphasize the idea that in ${\mathbf Z}_p$ the number $p$ is small and redo the previous computation with geometric series: $2/(1 - 3) = 2/(-2) = -1$. Show ${\mathbf Z}_3$ contains a square root of 7: $1 + 3 + 3^2 + 2 \cdot 3^4 + 2 \cdot 3^5 + \cdots$.
(To explain why $p$ being prime is important, say the $p$-adic integers form an integral domain, and for a contrast you could define $Z_{10}$ in a similar way and say there is a number $x$ in ${Z}_{10}$ besides 0 and 1 satisfying $x^2 = x$: $x = 5 + 2\cdot 10 + 6\cdot 10^2 + 9\cdot 10^4 + 8\cdot 10^5 + \cdots$. Compute the first few digits of $x^2$ to check this works. This is related to the elementary school question of finding integers whose square ends in themselves: $5^2$ ends in 5, $25^2$ ends in 25, $625^2$ ends in 625, and so on. The 10-adic solution of $x^2 = x$ which I wrote the initial expansion of above packages all of this information into one number.)
You could introduce a topology on ${\mathbf Z}_p$ where numbers are close if a long string of initial digits are the same and make a metric from this too. Then indicate how this makes ${\mathbf Z}_p$ compact by a sequential argument, in the same spirit in which $[0,1]$ is compact by an argument with decimal expansions. A key new feature here, which those with experience only in real and complex analysis haven't seen before, is that ${\mathbf Z}_p$ is a compact ring. In ordinary geometry there are plenty of compact groups, but no compact rings.
A nice application of this compactness is the finiteness of integral solutions to certain equations. For example, $x^2 - 7y^2 = 1$ has an infinite number of integral solutions, but $x^3 - 7y^3 = 1$ has just two integral solutions: (1,0) and (2,1). One way to prove this finiteness is to use $3$-adic power series. By algebraic methods one can show that if integers $x$ and $y$ satisfy $x^3 - 7y^3 = 1$ then $x - y\sqrt[3]{7} = (2-\sqrt[3]{7})^n$ for some integer $n$; the solutions $(x,y) = (1,0)$ and $(2,1)$ correspond to $n = 0$ and $n = 1$, respectively. When you expand $(2-\sqrt[3]{7})^n$ (say, by the binomial theorem when $n > 0$) you get a formula $a_n + b_n\sqrt[3]{7} + c_n\sqrt[3]{49}$ with integer coefficients $a_n, b_n, c_n$ and we want $c_n = 0$. That is a very strong constraint, since normally you don't expect $c_n = 0$. There is an exponential formula for $c_n$ in terms of $n$: $$ c_n = \frac{1}{21}\left(\sqrt[3]{7}(2 - \sqrt[3]{7})^n + \omega\sqrt[3]{7}(2 - \omega\sqrt[3]{7})^n + \omega^2\sqrt[3]{7}(2 - \omega^2\sqrt[3]{7})^n\right), $$ where $\omega$ is a cube root of unity. In the same way $a^x$ can be expanded as a real power series in $x$ when $a > 0$, the formula for $c_n$ above can be expanded into a $3$-adically convergent power series in $n$ by interpreting $\omega$ and $\sqrt[3]{7}$ to be a cube root of unity and a cube root of 7 in the 3-adics. (Strictly speaking you need to pass to a finite extension of the 3-adics to pick up a cube root of unity, but let's gloss over that point.) Asking for $c_n$ to be 0 is then asking for $n$ to be an integral root of a 3-adic power series. Just as a nonzero analytic function on a closed (hence compact) disc in ${\mathbf C}$ has finitely many roots in the disc, a 3-adic power series that converges on ${\mathbf Z}_3$ has finitely many roots in ${\mathbf Z}_3$. Since ${\mathbf Z}$ is inside of ${\mathbf Z}_3$ this implies in particular that there are finitely many roots in ${\mathbf Z}$. To make this result effective (i.e., to know $n=0$ and $n=1$ are the only 3-adic roots of that power series), you need techniques to bound the number of $3$-adic roots of a $3$-adic power series, and that goes beyond the scope of your talk. :) Details are written, for instance, in section 6.4.7 of Henri Cohen's "Number Theory I: Tools and Diophantine Equations". This technique of proving effective finiteness theorems for integral solutions of Diophantine equations by $p$-adic methods goes back to work of Strassman and Skolem and later developments in this direction are due to Chabauty and Coleman. However, even without an effective bound it's striking to see that the finiteness of integral solutions to an equation can be explained by an argument inspired by finiteness of zeros of a holomorphic function in a compact set, by sticking the integers into a compact domain on which a (3-adic) power series converges.
Here is a cute application of $p$-adic continuity of polynomials on $\mathbf Q$ (as a contrast to continuity of polynomials on $\mathbf Q$ in the "usual" topology your audience will know). It was the subject of a math.stackexchange question here. If we write out $\sqrt{1+x}$ as a power series it is $$ \sqrt{1+x} = \sum_{n \geq 0} \binom{1/2}{n}x^n = 1 + \frac{1}{2}x - \frac{1}{8}x^2 + \frac{1}{16}x^3 - \frac{5}{128}x^4 + \frac{7}{256}x^5 - \frac{21}{1024}x^6 + \cdots $$ and a striking feature is that the coefficients all have denominators that are powers of 2. It's not a surprise there is some power of 2 in the denominator considering a formula for the coefficient of $x^n$ is $$ \binom{1/2}{n} = \frac{(1/2)(1/2-1)(1/2-2)\cdots(1/2-n+1)}{n!}, $$ but why are there are no other primes in the denominator when you simplify the right side? One answer, which doesn't use anything $p$-adic, is to grind out the algebra and check that $$ \binom{1/2}{n} = \frac{(-1)^{n-1}}{2^{2n-1}}\left(\binom{2n-2}{n-1} - \binom{2n-2}{n}\right). $$ The binomial coefficients on the right are integers so the denominator is a power of 2. (That difference of binomial coefficients has a combinatorial interpretation as the $(n-1)$-th Catalan number.) Here is a slicker explanation of why the denominator is a power of 2: instead of directly seeing the power of 2 in the denominator of the rational number $\binom{1/2}{n}$, we will instead show for any prime $p$ other than 2 that $\binom{1/2}{n}$ is a $p$-adic integer, so it has no $p$ in its denominator as a reduced fraction. For odd $p$ we have the $p$-adic limit formula $\frac{1}{2} = \lim_{k \rightarrow \infty} \frac{p^k+1}{2}$, and the terms in that limit sequence are integers. The polynomial function $\binom{X}{n} \in {\mathbf Q}[X]$ is continuous in the $p$-adic topology, just as polynomials in ${\mathbf Q}[X]$ are continuous in the "usual" topology (in the reals), so by $p$-adic continuity $$ \binom{1/2}{n} = \lim_{k \rightarrow \infty} \binom{(p^k+1)/2}{n}. $$ Every binomial coefficient on the right is a positive integer for large $k$, so $\binom{1/2}{n}$ is a $p$-adic limit of integers and therefore is a $p$-adic integer. This means $\binom{1/2}{n}$ has no $p$ in its denominator. This holds for all odd $p$, so the only prime in the denominator of $\binom{1/2}{n}$ is 2.
The same basic idea shows for any nonzero rational number $r$ that the primes in the denominator of $\binom{r}{n}$ are limited to the primes in the denominator of $r$. For instance, the only primes in the denominator of $\binom{14/75}{n}$ are 3 and 5 because, with experience, one can see with $p$-adic limits that this fraction is $p$-adically integral for any $p$ other than 3 or 5. I doubt you're going to find a proof of that fact by some "explicit formula" method like the first proof I indicated for $\binom{1/2}{n}$ only having 2's in its denominator.