The project you've found is a (deliberate!) joke.
It is true that $\pi$ is suspected to be normal in all bases, which would imply that every finite sequence of hex digits appears somewhere (indeed, many times) in the hexadecimal expansion of $\pi$.
But this cannot be used for compression -- the trouble is that the number $A$ that tells you where to find your file in $\pi$ will -- in the vast majority of cases -- be so large that storing $A$ takes up even more space than it would take to store the original file.
The BBP formula is not particularly suited for finding a particular sequence in $\pi$, except by trial and error, or by starting somewhere in $\pi$ and keep producing digits until you randomly come across the sequence you're looking for. So at first, the goal of the project would be completely impossible -- just locating a ten-byte file would take lifetimes. (That is, some multiple of the lifetime of the universe).
The kicker is in this part of the description:
Now, we all know that it can take a while to find a long sequence of digits in $\pi$ so for practical reasons, we should break the files up into smaller chunks that can be more readily found.
In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in $\pi$.
So it doesn't actually "compress" anything -- it just stores, for each byte in the file, a position in $\pi$ where that particular byte can be found. (And finding such short a segment is certainly doable by brute force). But storing such a position takes more than a byte. So all in all it's just a simple substitution code, with a particularly inefficient implementation.
(And then there's some further joking around, claiming that the indices don't count as space used because they're "metadata".)
Note that $\pi=6\arcsin\left(\frac12\right)$. So, since$$\arcsin(x)=\sum_{n=0}^\infty \frac1{2^{2n}}\binom{2n}n\frac{ x^{2n+1}}{2n+1},$$you have$$\pi=\sum_{n=0}^\infty\frac6{2^{4n+1}(2n+1)}\binom{2n}n.$$Now, for each $N\in\mathbb{Z}^+$, let$$S_N=\sum_{n=0}^N\frac6{2^{4n+1}(2n+1)}\binom{2n}n\text{ and let }R_N=\sum_{n=N+1}^\infty\frac6{2^{4n+1}(2n+1)}\binom{2n}n.$$Then:
- $(\forall N\in\mathbb{Z}^+):\pi=S_N+R_N$;
- the sequence $(S_N)_{N\in\mathbb{Z}_+}$ is strictly increasing and $\lim_{N\to\infty}S_N=\pi$. In particular, each $S_N$ is a better approximation of $\pi$ than the previous one.
Since$$(\forall n\in\mathbb N):\binom{2n}n<4^n=2^{2n},$$you have$$R_N<\sum_{n=N+1}^\infty\frac6{2^{2n+1}}=\frac1{4^N}.$$So, taking $N=0$, you get that $\pi=S_0+R_0$. But $S_0=3$ and $R_0<1$. So, the first digit of $\pi$ is $3$. If you take $N=3$, then $\pi=S_3+R_3$. But $S_3\approx3.14116$ and $R_3<0.015625$. So, the second digit is $1$. And so on…
Best Answer
Since you allow any base, (16 in particular) and randomized algorithm, you can use the Bailey-Borwein-Plouffe formula which allows you to compute the $n^{th}$ digit of $\pi$, without having to compute the earlier $n-1$ digits! (Alas, such a algorithm seems to have been discovered only for base-16.)
All Alice needs to do is pick "some" random digits and compare.