I fished around in Google scholar and found so many examples that I don't feel like listing any of the links. Nonetheless, a clear picture emerges of an answer that I found a bit surprising: The notation $\mathbb{Z}_p$ for the $p$-adic integers evolved in three separate parts. I should also explain that the real science of etymology is about the evolution of words or notation, not just "when did it first happen".
The subscript notation not only for the $p$-adic integers, but more generally for $p$-adic completions, already appears in several papers in the 1930s and 1940s. For instance Carl Ludwig Siegel says in 1941, "$R$ is the field of rational numbers, $R_p$ the field of $p$-adic numbers, where $p$ denotes any prime number, $R_\infty$ the field of real numbers; moreover $J$ is the ring of integral numbers and $J_p$ the ring of $p$-adic integers". Of course, no one would use this notation today!
The use of $Z$ for the integers has a semi-separate history. I even found an old paper, but more recent than this one by Siegel, that used $Z$ for the integers but $R$ for the $p$-adic integers, with no subscript.
Generally the notation for $p$-adic integers and $p$-adic numbers standardized at $Z_p$ and $Q_p$ in the 1950s. Quite possibly Bourbaki, Algebra, deserves credit for standardizing $Z$ and $Q$ for integers and rationals.
Blackboard bold notation ($\mathbb{Z}$ and $\mathbb{Q}$) came last, at least in print. Despite its name, it's no longer obvious to me that blackboard bold actually first came from blackboards or from typewriters. It's sometimes also credited to Bourbaki, but this seems to be wrong. There is a historical account by Lee Rudolph (in comp.text.tex) that credits certain typewriter models in the 1960s for producing blackboard bold typography for the integers, etc. If that is where it started, then the notation seemed to catch on fairly quickly, although there were holdouts that used ordinary bold for decades after that. (But, before blackboard bold was fashionable, it wasn't even standard to make the set of integers bold $\mathbf{Z}$ instead of just $Z$.)
As an aside, the collision of notation between the $p$-adic integers and the integers mod $p$ is unfortunate. I really prefer to write $\mathbb{Z}/n$ for the integers mod $n$, because it is then written exactly as it reads. Also, partly since it is such a commonly used object, I see no need for extra parentheses, or an extra $\mathbb{Z}$, and certainly just using an $n$ subscript is bad. I'm optimistic that this notation is the way of the future and it would be an interesting separate question in history of notation.
(Sorry, I didn't see the entire string of comments before I wrote all of this. The comments make most of these remarks, but it seems useful to combine them into one historical summary.)
From Ross Street's An Australian conspectus of higher categories:
Kelly developed the theory of enriched categories describing enriched adjunction and introducing the variety of limit he called end. I later pointed out that Yoneda had used this concept in the special case of additive categories using an integral notation which Brian Day and Max Kelly adopted. Following this, Mac Lane discussed ends for ordinary categories.
The Day-Kelly paper referred to is
B. Day and G.M. Kelly, Enriched functor categories, Reports of the Midwest Category Seminar, III, Springer, Berlin, 1969, pp. 178–191.
whose introduction states
What we call ends and coends were introduced in the case $V = \text{Ab}$ by Yoneda; we borrow from him the "integral notation".
The Yoneda paper referred to is
Yoneda, N., On Ext and exact sequences. Jour. Fac. Sci. Univ. Tokyo 8 (1960), 507 - 576.
where we find written on page 546 (thanks to Francesco for the page reference)
We shall write $\int_C H$ for the integration $I$ of $H$, and $\int_C^\ast H$ for the cointegration $J$ of $H$.
where his "integration" is our "coend", and dually.
Best Answer
In an email to me dated 17 February 1992, Peter Freyd said:
When the diagram gets too complicated for the pullbacks to be rectangles, such as in the final chapter of my book Practical Foundations of Mathematics, I strongly recommend making them at least parallelograms. Then it is clear that pullback is acting as a functor that transforms one part of the diagram to another. In particular, in a type-theoretic setting pullback is substitution; whilst this has been known for a long time, Section 8.2 of the book actually proves it.
William Butler (had) proved some important results about monads, which you will find in "Toposes, Triples and Theories" by Barr and Wells (free TAC reprints copy).
Other very smart categorists who left academia to become schoolteachers include Christian Mikkelsen (who was the first to derive colimits from limits in an elementary topos) and Sjoerd Crans (who did weak higher dimensional category theory).