In Jackson's Classical Electrodynamics, section 1.5 he represents the electric field in terms of the gradient of a scalar:
$$
\begin{align}
\mathbf{E} &= -\nabla\Phi(x),\\
\Phi(x) &\equiv \frac{1}{4\pi\epsilon_0}\int\frac{\rho(\mathbf{x}')}{|\mathbf{x}-\mathbf{x}'|}\,d^3x'
\end{align}
$$
and then states "where the integration is over all charges in the universe, and $\Phi$ is arbitrary only to the extent that a constant can be added to the right-hand side of (1.17)" (emphasis mine, 1.17 is the second equation posted above).
I don't understand why he states that $\Phi$ is arbitrary to a constant. Sure, the integral appears to be written as an indefinite integral, but the fact that it is an integration over all space indicates it is actually an improper integral with three-dimensional limits of integration of $\pm\infty$. In the improper integral, the constant of integration should cancel out and therefore not depend on some arbitrary constant.
Is this an error in the text (I didn't see it mentioned in any of the errata I could track down), or have I misunderstood something, or misinterpreted why this constant appears in the scalar potential?
I've tracked down the source of my original confusion. However, correcting for this is leading to a bit of additional confusion.
The original motivation for writing the electric field as the gradient of a scalar potential is that in the equation for the electric field (here given in terms of the charge density),
$$
\begin{equation}
\mathbf{E}(\mathbf{x}) = \frac{1}{4\pi\epsilon_0} \int_{\mathbb{R}^3} \rho(\mathbf{x}')
\frac{\mathbf{x}-\mathbf{x}'}
{\left|\mathbf{x}-\mathbf{x}'\right|^3}\, d^3x',
\end{equation}
$$
we can use the equation
$$
\begin{equation}
\frac{\mathbf{x}-\mathbf{x}'}{|\mathbf{x}-\mathbf{x}'|^3}
= – \nabla \left(\frac{1}{|\mathbf{x}-\mathbf{x}'|} + C\right),
\end{equation}
$$
to rewrite this in terms of a gradient. Jackson omitted the constant $C$ here, bringing it back up in the discussion of the scalar potential. If we carry $C$ through the remainder of the derivation we arrive at
$$
\begin{align}
\mathbf{E}(\mathbf{x}) &= -\nabla \Phi(\mathbf{x})\\
\Phi(\mathbf{x}) &\equiv \frac{1}{4\pi\epsilon_0} \int_{\mathbb{R}^3}
\left(\frac{\rho(\mathbf{x}')}{|\mathbf{x}-\mathbf{x}'|}+C\right)\, d^3x'.
\end{align}
$$
So, now the scalar potential correctly reflects the fact that it contains an arbitrary constant. It would be more useful to pull this constant out of the integral to represent the scalar potential as the integral plus some arbitrary offset. Since it's a constant that's easy:
$$
\begin{align}
\Phi(\mathbf{x}) &= \Phi_0 + \frac{1}{4\pi\epsilon_0} \int_{\mathbb{R}^3}
\left(\frac{\rho(\mathbf{x}')}{|\mathbf{x}-\mathbf{x}'|}\right)\, d^3x',\\
\Phi_0 &\equiv \frac{C}{4\pi\epsilon_0} \int_{\mathbb{R}^3} d^3 x'.
\end{align}
$$
The difficulty with this is that if we are truly integrating over all space then $\int_{\mathbb{R}^3} d^3 x'=\infty$ and $\Phi_0=0,\pm\infty$. The resolution here is probably as the the answer and comments suggest, that we can never measure the absolute value of the potential anyway and so this doesn't matter.
Best Answer
We don’t measure $\Phi$. We measure $\mathbf{E}$.
Imagine you are doing an experiment inside of a cavity in a conductor. There’s no field inside of your conductor, so all of your electric fields in your experiment are generated locally. (A practical implementation of this is a “Faraday cage.”)
Now, somebody outside of your experiment charges up your conductor. All of the excess charge moves to the exterior surface of the conductor; it has zero effect on the fields in your cavity. Your experiment proceeds in exactly the same way, because $\mathbf{E}$ is the same, even though $\Phi$ has changed.
It’s a useful convention to define $\Phi$ so that $\Phi\to 0$ in the limit where you are very far from any nonzero charge distribution. But for describing dynamics, we don’t care about that convention. We only care about $\nabla\Phi$.