(Note: I have edited the question to make my confusion more specific.)
In Huybrechts' Complex Geometry, on page 63 it is said that,
$\mathcal{I}_Y$ is the ideal sheaf of holomorphic functions vanishing on $Y$
while on page 84 the following is said,
Let $Y\subset X$ be an irreducible hypersurface $\ldots$ the ideal sheaf $\mathcal{I}$ of $Y$ and the subsheaf $\mathcal{O}(-Y)\subset\mathcal{O}_X$ $\ldots$ are equal.
Additionally on page 83 we have the following proposition.
Let $0 \neq s \in H^0(X,L)$. Then the line bundle $\mathcal{O}(Z(s))$ is isomorphic to $L$.
So in particular, at least for an effective divisor $D$, the sections of the line bundle $\mathcal{O}(D)$ vanish on $D$. But above we saw that $\mathcal{O}(-D)$ is the sheaf of holomorphic functions vanishing on $D$, so I would naively expect that $\mathcal{O}(-D)$ would be the bundle whose sections vanish on $D$.
How is it that $\mathcal{O}(-D)$ corresponds to the sheaf of functions vanishing on $D$, while it is $\mathcal{O}(D)$ that is the bundle whose sections vanish on $D$?
In particular it would be nice if someone could explicitly describe, in terms of functions on patches (as far as this can be done), $\mathcal{O}(p)$ and $\mathcal{O}(-p)$, for a point $p$ in $\mathbb{P}^1$, showing explicitly how they behave as bundles with or without global sections, and making clear why it is $\mathcal{O}(-p)$ and not $\mathcal{O}(-p)$ that can be seen as a subsheaf of the sheaf of regular functions (which in particular is the ideal sheaf of the point).
Best Answer
Your definition of $\mathcal O(D)$ seems to lacking precision. If $D$ is a single point $-p$, then $\mathcal O(D)$ is the sheaf of functions whose divisor is greater than $p$. Which means these functions must have a zero at $p$. If $D=p$, then $\mathcal O(D)$ is the sheaf of functions whose divisor is greater than $-p$, which means these functions can have a pole at $p$. So the point is signs matter. $\mathcal O(-Y)$ is correct.
Put $X = \mathbb P^1$.
$\mathcal O(-p)$ is defined to be the sheaf of holomorphic functions $f$ on $X$ such that the divisor of $f$ plus $-p$ is greater than $0$. This means sections of $\mathcal O(-p)$ must have a simple zero at $p$, and no poles elsewhere. It is clear that sections of $\mathcal O(-p)$ (which are holomorphic functions with the above condition) also are sections of $\mathcal O$ (whose sections are holomorphic functions, without any qualification).
$\mathcal O(p)$ is defined to be the sheaf of holomorphic functions $f$ on $X$ such that the divisor of $f$ plus $p$ is greater than $0$. This means sections of $\mathcal O(p)$ are allowed to have poles as I said. For example $1/(z-p)$ is a global section of $\mathcal O(p)$, but this is certainly not a section on $\mathcal O$.