If the map from $X$ to the maxspec of $\mathcal O(X)$ is a bijection, then $X$ is indeed affine.
Here is an argument:
By assumption $X \to $ maxspec $\mathcal O(X)$ is bijective, thus quasi-finite,
and so by (Grothendieck's form of) Zariski's main theorem, this map factors as an open embedding of $X$ into a variety that is finite over maxspec $\mathcal O(X)$.
Any variety finite over an affine variety is again affine, and hence $X$ is an open subset of an affine variety, i.e. quasi-affine. So we are reduced to considering the case when $X$ is quasi-affine, which is well-known and straightforward.
(I'm not sure that the full strength of ZMT is needed, but it is a natural tool
to exploit to get mileage out of the assumption of a morphism having finite fibres, which is what your bijectivity hypothesis gives.)
In fact, the argument shows something stronger: suppose that we just assume
that the morphism $X \to $ maxspec $\mathcal O(X)$ has finite non-empty fibres,
i.e. is quasi-finite and surjective.
Then the same argument with ZMT shows that $X$ is quasi-affine. But it is standard that the map $X \to $ maxspec $\mathcal O(X)$ is an open immersion when $X$ is quasi-affine,
and since by assumption it is surjecive, it is an isomorphism.
Note that if we omit one of the hypotheses of surjectivity or quasi-finiteness, we can find a non-affine $X$ satisfying the other hypothesis.
E.g. if $X = \mathbb A^2 \setminus \{0\}$ (the basic example of a quasi-affine,
but non-affine, variety), then maxspec $\mathcal O(X) = \mathbb A^2$, and the open immersion $X \to \mathbb A^2$ is evidently not surjective.
E.g. if $X = \mathbb A^2$ blown up at $0$, then maxspec $\mathcal O(X) =
\mathbb A^2$, and $X \to \mathbb A^2$ is surjective, but has an infinite fibre
over $0$.
Caveat/correction: I should add the following caveat, namely that it is not always true, for a variety $X$ over a field $k$, that $\mathcal O(X)$ is finitely generated over $k$, in which case maxspec may not be such a good construction to apply, and the above argument may not go through. So in order to conclude that $X$ is affine, one should first insist that $\mathcal O(X)$ is finitely generated over $k$, and then that futhermore the natural map $X \to $ maxspec $\mathcal O(X)$ is quasi-finite and surjective.
(Of course, one could work more generally with arbitrary schemes and Spec rather than
maxspec, but I haven't thought about this general setting: in particular, ZMT requires some finiteness hypotheses, and I haven't thought about what conditions might guarantee that the map $X \to $ Spec $\mathcal O(X)$ satisfies them.)
Incidentally, for an example of a quasi-projective variety with non-finitely generated ring of regular functions, see this note of Ravi Vakil's
1) $X$ is irreducible and $D(f)$ is open in $X$ implies that $D(f)$ is also irreducble. Now, since $j: Y \rightarrow D(f)$ is continuous and bijective, it is a homeomorphism. This implies that $Y$ is also irreducible (as it is topologically homeomorphic to $D(f)$).
2) You can check that whenever you have a homeomorphism between two topological spaces, the inverse map (theoretical) is also continuous, hence, is also a homeomorphism.
3) To show that $j$ is an isomorphism of ringed spaces, you need to construct $j^{\#}: \mathcal{O}_{D(f)} \rightarrow j_{\ast} \mathcal{O}_Y$ and verify that it gives you an isomorphism of sheaves for each open subset $U \subset D(f)$. This can be done as follows. For any section $g \in \mathcal{O}_{D(f)}(U)$ that is represented by $G \in k[T_1, ...., T_n]$, we define its image to be the section represented by $H(T_1,...,T_n, T_{n+1}) = G(T_1,...,T_n)$. There is a little bit detailed and routine checking required here that $H$ does represent validly a unique section in $j_{\ast \mathcal{O}_Y(U)} = \mathcal{O}_Y(j^{-1}(U))$ and vice versa, i.e. a section in the $\mathcal{O}_Y(j^{-1}(U))$ has a preimage in $\mathcal{O}_{D(f)}(U)$. It probably uses some fact in step 1 but all in all, it is not too bad.
Best Answer
1) Standard open sets are defined for every locally ringed space. If $f \in \Gamma(X,\mathcal{O}_X)$, then $X_f$ (sometimes also called $D(f)$) is by definition the set of all $x \in X$ such that $f_x \notin \mathfrak{m}_x$, where $\mathfrak{m}_x$ is the maximal ideal of the local ring $\mathcal{O}_{X,x}$. Equivalently, $f(x) \neq 0$ in the residue field $k(x) = \mathcal{O}_{X,x}/\mathfrak{m}_x$. This is also the reason why often $X_f$ is called the "locus where $f$ doesn't vanish" or "where $f$ is invertible". It is an easy exercise to show that $X_f$ is in fact open, and that we have the standard identities $X_1 = X$, $X_f \cap X_g = X_{fg}$. When $X$ is an affine algebraic set, this coincides with the locus where $f$ doesn't vanish defined in the usual sense (and probably this is what you meant by $D(f) \subseteq V$).
2) Again quasi-coherent sheaves make sense for arbitrary ringed spaces. And it is a very bad idea to give definitions only for algebraic sets $\subseteq \mathbb{A}^n$ and try to extend them via chosen isomorphisms! You should work with intrinsic geometric objects instead, and (locally) ringed spaces provide a nice framework for that. So let's use this language.
A quasi-coherent module on a ringed space $X$ is just a module $M$ (i.e. what most people call a sheaf of modules) on $X$ such that locally on $X$ there is a presentation $\mathcal{O}^{\oplus I} \to \mathcal{O}^{\oplus J} \to M \to 0$. So to be more precise: There is an open covering $X = \cup_i X_i$ such that for each $i$ there is an exact sequence (which, of course, does not belong to the data) $\mathcal{O}|_{X_i}^{\oplus I} \to \mathcal{O}|_{X_i}^{\oplus J} \to M|_{X_i} \to 0$. Quasi-coherent modules constitute a (tensor) category $\mathrm{Qcoh}(X)$, which is by the way a very interesting and deep invariant of $X$, especially when $X$ is a variety.
How to construct quasi-coherent modules on a ringed space $X$? Well pick a $\Gamma(X,\mathcal{O}_X)$-module $M$. Then I claim that we can construct a quasi-coherent module $\tilde{M}$ on $X$ as follows: Choose a presentation $\Gamma(X,\mathcal{O}_X)^{\oplus I} \to \Gamma(X,\mathcal{O}_X)^{\oplus J} \to M \to 0.$ Represent the morphism on the left as a "relation matrix" consisting of elements of $\Gamma(X,\mathcal{O}_X)$. Now, every such global section corresponds to a homomorphism $\mathcal{O}_X \to \mathcal{O}_X$. Thus we can produce a matrix consisting of endomorphisms over $\mathcal{O}_X$, and thus a morphism $\mathcal{O}_X^{\oplus I} \to \mathcal{O}_X^{\oplus J}$. Define $\tilde{M}$ to be the cokernel. By definition, this is quasi-coherent! This already produces lots of examples; in fact all $X$ is an affine variety, but only few if $X$ is projective.
To give a more concise definition which does not depend on the presentation: Just define $\tilde{M}$ to be the sheaf associated to the presheaf $U \mapsto \Gamma(U,\mathcal{O}_X) \otimes_{\Gamma(X,\mathcal{O}_X)} M$. This definition easily implies a more conceptual characterization of the functor $M \to \tilde{M}$ from $\Gamma(X,\mathcal{O}_X)$-modules to quasi-coherent modules modules on $X$: It is left adjoint to the global section functor! In fact, everything you want to know about $\tilde{M}$ already follows from this adjunction. You may forget about the details of the construction, you just have to remember $\hom(\tilde{M},F) \cong \hom(M,\Gamma(X,F))$, which actually holds for every module $F$ on $X$.
So what happens when $X$ is some affine variety? Then the sets $X_f$ constitute a basis for the topology of $X$, and we have $\Gamma(X_f,\mathcal{O}_X) = \Gamma(X,\mathcal{O}_X)_f$. Namely, this is well-known if $X \subseteq \mathbb{A}^n$ and then generalizes immediately to affine varieties, which are isomorphic as ringed spaces to such concrete varieties. Let $M$ be a $\Gamma(X,\mathcal{O}_X)$-module. Now it turns out that the presheaf defined above is actually a sheaf! This comes down to the following: If $f_1,\dotsc,f_n \in \Gamma(X,\mathcal{O}_X)$ generate the unit ideal (i.e. the corresponding sets $X_{f_i}$ cover $X$), then the canonical sequence
$$0 \to M \to \prod_{i} M_{f_i} \to \prod_{i,j} M_{f_i f_j}$$
is exact. Everyone should have done this proof instead of looking it up in the standard sources. Because I think it is quite enlightening and in fact purely geometric if you think of $f_1,\dotsc,f_n$ as a partition of unity.
Anyway, so this tells us that we don't need associated sheaves in the definition of $\tilde{M}$. Thus, by definition, on the open subset $X_f$ it is given by
$$\Gamma(X_f,\tilde{M}) = \Gamma(X_f,\mathcal{O}_X) \otimes_{\Gamma(X,\mathcal{O}_X)} M = \Gamma(X,\mathcal{O}_X)_f \otimes_{\Gamma(X,\mathcal{O}_X)} M = M_f.$$
So this describes some quasi-coherent sheaves on affine varieties. In fact, one can show that every quasi-coherent sheaf on an affine variety $X$ has the form $\tilde{M}$. Namely, one shows that for every such sheaf $F$ the canonical counit morphism of the adjunction mentioned above $\tilde{\Gamma(X,F)} \to F$ is an isomorphism. Again, this is a very nice exercise. After some thought you will see that this is just another application of the exact sequence above. So this provides, for every affine variety, an equivalence of categories
$$\mathrm{Qcoh}(X) \cong \mathrm{Mod}(\Gamma(X,\mathcal{O}_X)).$$
By the way, if you define $\tilde{M}$ on an affine variety by $\Gamma(X_f,\tilde{M}) = M_f$ and extended via projective limits to arbitrary open subsets, then you probably would like to know that this is a sheaf. And again this comes down to the exact sequence above. You cannot get around it. I don't like this approach because it is somewhat clumsy, you don't get the general picture, and it doesn't produce a formula for $\tilde{M}(U)$ for arbitrary $U$. Therefore I've chosen the rather abstract but hopefully concise approach above. Of course nothing is new, you can find all that in EGA I, the Stacks Project, etc.