The elements of $\mathscr{O}_{\mathbb{A}^n,a}$ are just rational functions $\frac{g}{f}$, where $f, g$ are polynomials of $n$ variables (i.e. elements of $\mathscr{O}_{\mathbb{A}^n}$) such that $f(a) \neq 0$.
This of course is nothing but a copy of Lemma 3.21 in your linked notes.
Intuitively, you should think about $\mathbb{A}^n$ as the $n$-dimensional complex space $\mathbb{C}^n$.
A polynomial $f\in \mathbb{C}[X_1, \cdots, X_n]$ then defines a function on $\mathbb{C}^n$: it sends a point $a = (a_1, \cdots, a_n)$ to $f(a_1, \cdots, a_n)$.
Now if $f, g\in \mathbb{C}[X_1, \cdots, X_n]$ are two polynomials, in general you cannot say that $\frac{g}{f}$ defines a function on $\mathbb{C}^n$, simply because the value of $f$ may vanish at some points.
But for a given point $a$, if we have $f(a) \neq 0$, then it's clear that $f$ does not vanish in a small neighborhood of $a$ (with respect to the usual topology of $\mathbb{C}^n$), hence on that small neighborhood we may define $\frac{g}{f}$ without problem.
And if we gather all functions that can be defined as $\frac{g}{f}$ in a small neighborhood of the given point $a$, we get the local ring $\mathscr{O}_{\mathbb{A}^n,a}$.
I am also unsure of my solution to this problem, so any corrections are welcome! I think you are on the right track, the isomorphism must be the identity map. The only thing that is problematic is showing that the sheaves of regular function are the same.
The question calls for us to view $U \cap Y$ in two ways. First, we view $U \cap Y$ as a closed subset of $U$. In this case, the sheaf $O_{U \cap Y}$ is by definition: For any $V$ open in $U \cap Y$, $O_{U \cap Y} (V) = \{ \varphi : V \to K :$ for all $a \in V$ there exists an open neighborhood $ N$ of $U$ containing $a$, and $\psi \in O_{U}(N)$ with $\varphi = \psi$ on $V \cap N \}$. Then, we note that $O_U(N) = O_X(N) |_U = O_X(N)$ where the second equality is because $N \subset U$.
On the other hand, we can view $U \cap Y$ as an open subset of $Y$. Then for any $V$ open subset of $U \cap Y$, we have that the sheaf is just the restriction: $O_{U \cap Y}(V) = O_{Y} (V)$. Lets apply the definition of $O_{Y} (V')$. By definition, for any $V'$ open in $Y$, $O_Y(V') = \{ \varphi : V' \to K : $ for all $a \in V'$ there exists an open neighborhood $N'$ of $X$ containing $a$, and $\psi$ in $O_X(N')$ with $\varphi = \psi$ on $V' \cap N' \}$.
Then just need to show that the restriction of the last set to $U \cap Y$ is equal to the first set. Since $V'$ is going to be an open subset of $U \cap Y$, we can replace "$V'$ open in $Y$" with "$V'$ open in $U \cap Y$".
$N'$ is an open subset of $X$, and $N' \subset U \cap Y$, so $N'$ is an open neighborhood of $U$ also. So we can replace "open neighborhood $N'$ of $X$" with "open neighborhood $N'$ of $U$". Then both sets can be seen to be equal.
Best Answer
If you're trying to define what it means for a function to be regular on an open subset of an affine variety, you must you definition 2: a function is regular on this open subset iff it can locally be written as a ratio of polynomials with non-vanishing denominators.
In the special case where you're defining what it means for a function to be regular on the entire affine variety, this is equivalent to definition 1: a function is regular on the entire affine variety iff it can globally be written as a polynomial. And yes, I do mean "globally", not "locally".