I'm somewhat embarrassed to be confused about this issue after a while studying algebraic geometry, but here goes.
Let $\iota: Y \hookrightarrow X$ be the inclusion of a closed subscheme, $U$ is the complement of $Y$ in $X$, and $\mathcal{F}$ a sheaf on $X$. Then one has an exact sequence of sheaves
$$0 \rightarrow \mathcal{F}_U \rightarrow \mathcal{F} \rightarrow \mathcal{F}_Y \rightarrow 0.$$
Here $\mathcal{F}_Y = \iota_* (\iota^{-1} \mathcal{F})$ and $\mathcal{F}_U = j_{!} (j^{-1}\mathcal{F})$ where $j: U \hookrightarrow X$ is the natural open embedding.
On the other hand, one has the exact sequence of the closed subscheme
$$ 0 \rightarrow \mathcal{I}_Y \rightarrow \mathcal{O}_X \rightarrow \iota_* \mathcal{O}_Y \rightarrow 0.$$
If $\mathcal{F}$ is locally free, then we can tensor this up to get the sequence
$$ 0 \rightarrow \mathcal{F} \otimes \mathcal{I}_Y \rightarrow \mathcal{F} \rightarrow \mathcal{F} \otimes \iota_* \mathcal{O}_Y \rightarrow 0.$$
I had always thought, and even denoted, both the sheaves $\mathcal{F}_Y$ and $\mathcal{F} \otimes \iota_* \mathcal{O}_Y$ as $\mathcal{F}|_Y$ (note that $\mathcal{F} \otimes \iota_* \mathcal{O}_Y = \iota_* (\iota^* \mathcal{F} \otimes \mathcal{O}_X)$, but I guess they can't be equal since e.g. $\mathcal{F}_U \neq \mathcal{F}(-Y)$ – one is a vector bundle and the other isn't. Somewhere in all the sheafification stuff, I've lost my head and I'm
seeking some intuition about the difference between $\mathcal{F}_Y$ and $\mathcal{F} \otimes \iota_* \mathcal{O}_Y$. For instance, continuing to assume that $\mathcal{F}$ is a vector bundle, the second sheaf (I think) should correspond to the restriction of the vector bundle to $Y$; what is the first sheaf?
Best Answer
For any sheaf, $i^{-1}\mathcal F$ doesn't change the stalks. You just take the stalks of $\mathcal F$, but only pay attention to them at the points of $y$. So if $x$ is a closed point in $X$ and $i: x \hookrightarrow X$ is the embedding, $i^{-1} \mathcal O_X$ is literally the stalk at $x$, i.e. $\mathcal O_{X,x}$.
This is quite a bit different from $i^*\mathcal O_{X,x}$, which is the residue field at $x$.
As you guessed, $i^*$ applied to locally free sheaves corresponds to restricting vector bundles. The $i^{-1}$ operation isn't so geometrically meaningful for coherent sheaves (other than as a step on the way to defining $i^*$), but is important for more topological considerations (e.g. if we were working with locally constant or constructible sheaves on the complex topology of a complex variety, or on the etale topology of a scheme; it also comes up in the proof of Grothendieck's vanishing theorem, which is of a purely topological nature).