Let $p_n(\vec{x})$ be the probability that a symmetric random walk (SRW) on $\mathbb{Z}^n$ starting at the origin reaches the point $\vec{x}\in\mathbb{Z}^n-\{0\}$ before returning to the origin. What is $p_2(\vec{x})$?
Judging from this post, I fear the answer might be too complicated, but it's worth a shot. I've already been able to calculate $p_1(x)=\frac{1}{2|x|}$ as follows:
Let $p_n(\vec{s},\vec{x})$ be the probability that a SRW on $\mathbb{Z}^n$ starting at $\vec{s}$ reaches the point $\vec{x}\in\mathbb{Z}^n-\{0\}$ before the origin.
Lemma. $$p_n(\vec{s},\vec{x})+p_n(\vec{x}-\vec{s},\vec{x})=1\quad\text{for}\quad n=1,2\tag{1}$$
Proof. Firstly, a SRW for $n=1,2$ will reach each point with probability $1$. Hence, we may assume WLOG that a SRW will either reach $0$ first, or $\vec{x}$ first. If $W$ is a walk, then $W+\vec{s}$ reaches $\vec{x}$ before $0$ iff $-W+\vec{x}-\vec{s}$ reaches $0$ before $\vec{x}$. Hence, $p_n(\vec{s},\vec{x})=1-p_n(\vec{x}-\vec{s},\vec{x})$. Rearranging gives the desired equality $(1)$. \begin{multline}\shoveright\square
\end{multline}
Theorem. $$p_1(x)=\frac{1}{2|x|}\tag{2}$$
Proof. Suppose $0<s<x$. Then a SRW that reaches $x$ before $0$ will also reach $s$ before $0$. Then, $$p_1(s)p_1(s,x)=p_1(x)\tag{3}$$
Then we may write $(1)$ as
$$\frac{1}{p_1(s)}+\frac{1}{p_1(x-s)}=\frac{1}{p_1(x)}\tag{4}$$
Setting $s=1$ and using induction gives
$$\frac{1}{p(x)}=\frac{x}{p_1(1)}\tag{5}$$
We can quickly check that $p_1(1)=\frac{1}{2}$: either the first step in the walk is to $1$, or the first step is to $-1$, in which case the walk will have to return to the origin before reaching $1$. Hence, $p_1(x)=\frac{1}{2x}$. By symmetry, $p_1(-x)=p_1(x)$, so we arrive at $(2)$.\begin{multline}\shoveright\square
\end{multline}
I've tried ways of extending this method to $n=2$, but to no avail. The biggest problem is that there doesn't seem to be a nice analogue of $(3)$ for $n=2$ because a walk that reaches $\vec{x}$ does not necessarily pass through any other specific point. According to Wikipedia, $(2)$ can be derived from the fact that a SRW on $\mathbb{Z}^n$ is a martingale, but I know nothing of martingales so I do not know how to use this information to find a formula for $n=2$.
EDIT:
I should have looked a little more closely at the post I linked. There is a link in the comments leading to this page, which shows that if we define $R_{n,m}$ to be the resistance between the origin and the point $(n,m)$ on $\mathbb{Z}^2$ where every edge has unit resistance, then
$$R_{1,0} = \frac{1}{2}\\ R_{m,m}=\frac{2}{\pi}\sum_{k=1}^m\frac{1}{2k-1}\quad\quad m>0\\ 4R_{n,m} -R_{n-1,m}-R_{n+1,m}-R_{n,m-1}-R_{n,m+1}=0\quad\quad (n,m)\neq (0,0)$$
Using symmetry accross the $x$-axis, $y$-axis, and diagonals $y=\pm x$, the above is enough to determine $R_{n,m}$ for every $n,m$. Then,
$$\boxed{p_2((n,m))=\frac{1}{4R_{n,m}}}$$
As a sanity check, the asymptotic from Yuval Peres's answer gives
$$p_2((m,m))=\frac{\pi}{8}\left(\sum_{k=1}^m\frac{1}{2k-1}\right)^{-1}\sim \left(\frac{4}{\pi}\log(m)+\frac{4\gamma}{\pi}+\frac{8}{\pi}\log 2\right)^{-1}\quad\quad m>0$$
which is a tight approximation easily derivable from the fact that $H_{m-1}-\gamma =\psi(m) \sim \log m$, and $\sum_{k=1}^m\frac{1}{2k-1}=H_{2m-1}-\frac{1}{2}H_{m-1}$.
Best Answer
I will use the language of electrical networks, as presented in [1],[2],[3]. For $x \in \mathbb Z^2$, the probability $p_2(x)$ is known as the escape probability, and it is a quarter of the effective conductance between $0$ and $x$ when every edge of the lattice has unit resistance. The effective resistance is the voltage difference between $0$ and $x$ when a unit of current is sent from $x$ to $0$. Let $a(\cdot): \mathbb Z^2 \to [0,\infty)$ be the potential kernel studied in [4], page 124. It is shown there that $$a(x)=\frac2{\pi} \log|x|+ \frac{2\gamma}{\pi} +\frac3{\pi}\log(2)+O(|x|^{-2})\,.$$
Then the voltage function when a unit of current is sent from $x$ to $0$ is $\psi(v)=\frac14[a(v)-a(v-x)]$. The voltage difference is $\psi(x)-\psi(0)= \frac12a(x).$ Thus the escape probability from $0$ to $x$ is $$p_2(x)=\frac1{2a(x)} \,.$$
Remark: In the one-dimensional case, $a(x)=|x|$ and the formula above applies as well for $p_1(x)$.
[1] Doyle, P.G. and Snell, J.L., 1984. Random walks and electric networks (Vol. 22). American Mathematical Soc. https://arxiv.org/pdf/math/0001057
[2] Levin, David A., and Yuval Peres. Markov chains and mixing times. Vol. 107. American Mathematical Soc., 2017. https://www.yuval-peres-books.com/markov-chains-and-mixing-times/ Chapter 9
[3] Lyons, Russell, and Yuval Peres. Probability on trees and networks. Vol. 42. Cambridge University Press, 2017. https://www.yuval-peres-books.com/probability-on-trees-and-networks/ Chapter 2
[4] Spitzer, Frank. Principles of random walk. Vol. 34. Springer Science & Business Media, 2001.