The same reason we study abstract groups instead of their embeddings into symmetry groups. If we know some properties of a manifold $M$ which are intrinsic and don't depend on embeddings, and we later encounter that manifold $M$ somewhere else in a different embedding than the one we're used to, then we automatically know a list of things we can say about this new copy of $M$. The point is that the study of things like groups and manifolds naturally separates into the study of their abstract structure and the study of their concrete representations, and failing to make this separation is unnecessarily confusing.
For example, topological and smooth manifolds don't have a notion of length of paths or volume: these things are dependent on an embedding into $\mathbb{R}^n$, so if I meet a manifold in a different embedding I can't assume that things like length and volume are the same, and I will only get confused if I do assume this. I can assume, however, that things like the number of connected components, the homotopy groups, the cohomology, etc. are the same, so whenever I meet a manifold whose cohomology groups I know I can always apply that information.
I wouldn't say that the abstraction is "necessary." Nothing in mathematics is necessary. We make the definitions we make so we can more easily understand certain mathematical phenomena, and the phenomena we are interested in with regard to manifolds are easier to understand if we define abstract manifolds first and then worry about their embeddings later.
You get the uniqueness result if the space is Hausdorff.
Let $\langle X,\tau\rangle$ be a compact space. Suppose that $p\in X$ is in the closure of $Y=X\setminus\{p\}$, and let $\tau_Y$ be the associated subspace topology on $Y$; $\langle X,\tau\rangle$ is then a compactification of $\langle Y,\tau_Y\rangle$.
Suppose that $p\in U\in\tau$, and let $V=U\cap Y$. Then $\varnothing\ne V\in\tau_Y$, so $Y\setminus V$ is closed in $Y$. Moreover, $Y\setminus V=X\setminus U$ is also closed in $X$, which is compact, so $Y\setminus V$ is compact. That is, every open nbhd of $p$ in $X$ is the complement of a compact, closed subset of $Y$. Thus, if $\tau'$ is the topology on $X$ that makes it a copy of the Alexandroff compactification of $Y$, then $\tau\subseteq\tau'$.
Now let $K\subseteq Y$ be compact and closed in $Y$, and let $V=Y\setminus K\in\tau_Y$. If $X\setminus K=V\cup\{p\}\notin\tau$, then $p\in\operatorname{cl}_XK$. If $X$ is Hausdorff, this is impossible: in that case $K$ is a compact subset of the Hausdorff space $X$ and is therefore closed in $X$. Thus, if $X$ is Hausdorff we must have $\tau=\tau'$, and $X$ is (homeomorphic to) the Alexandroff compactification of $Y$.
If $X$ is not Hausdorff, however, we can have $\tau\subsetneqq\tau'$. A simple example is the sequence with two limits. Let $D$ be a countably infinite set, let $p$ and $q$ be distinct points not in $D$, and let $X=D\cup\{p,q\}$. Points of $D$ are isolated. Basic open nbhds of $p$ are the sets of the form $\{p\}\cup(D\setminus F)$ for finite $F\subseteq D$, and basic open nbhds of $q$ are the sets of the form $\{q\}\cup(D\setminus F)$ for finite $F\subseteq D$. Let $Y=D\cup\{q\}$. Then $Y$ is dense in $X$, and $X$ is compact, and $Y$ itself is a closed, compact subset of $Y$ whose complement is not open in $X$.
Improved example (1 June 2015): Let $D$ and $E$ be disjoint countably infinite sets, let $p$ and $q$ be distinct points not in $D\cup E$, let $X=D\cup E\cup\{p,q\}$, and let $Y=D\cup E\cup\{q\}$. Points of $D\cup E$ are isolated. Basic open nbhds of $q$ are the sets of the form $\{q\}\cup (E\setminus F)$ for finite $F\subseteq E$, and basic open nbhds of $p$ are the sets of the form $\{p\}\cup\big((D\cup E)\setminus F\big)$ for finite $F\subseteq D\cup E$. Then $Y$ is a non-compact dense subspace of the compact space $X$, so $X$ is a (non-Hausdorff) compactification of $Y$. Let $K=\{q\}\cup E$. Then $K$ is a compact closed subset of $Y$, but $X\setminus K=\{p\}\cup D$ is not open in $X$.
(This avoids the question of whether it’s legitimate to look at the Alexandrov compactification of a compact space.)
Best Answer
Just to take this question from the "unanswered" list. (It was actually answered in comments.)
(1). The simplest example of a manifold which is not homeomorphic to an open subset of a compact manifold is an infinite disjoint union of circles. [Edit: I stand corrected: the simplest example is ${\mathbb N}$ with discrete topology. I was only thinking about manifolds of positive dimension.]
(2). If you want a connected example, then it first appears in dimension 2: If a connected surface $S$ has infinite genus then it is not homeomorphic to an open subset of a compact surface. This is intuitively clear, but I will nevertheless give a proof which works in all dimensions.
Since $S$ has infinite genus, the image of the natural map $$ \phi: H^1_c(S; {\mathbb R})\to H^1(S; {\mathbb R}) $$ has infinite rank (each "handle" in $S$ contributes a 2-dimensional subspace). Suppose that $S\to T$ is an open embedding of $S$ to a compact surface $T$. Then we have the commutative diagram $$ \begin{array}{ccc} H^1_c(S; {\mathbb R}) & \stackrel{\phi}{\to} & H^1(S; {\mathbb R})\\ \psi\downarrow & ~ & \eta\uparrow \\ H^1_c(T; {\mathbb R}) & \stackrel{\cong}{\to} & H^1(T; {\mathbb R}) \end{array} $$
(Note that the induced maps of ordinary and of compactly supported cohomology groups go in opposite directions, this is what used in the proof.) Since $H^1(T, {\mathbb R})$ is finite-dimensional, the image of $\eta\circ \psi$ is also finite-dimensional, which is a contradiction.
Edit. There are less trivial examples in dimension 3. Haken proved that a certain open contractible 3-manifold does not embed in any compact 3-manifold:
W. Haken, Some results on surfaces in 3-manifolds, Studies in Modern Topology, Math. Assoc. Amer. (distributed by Prentice-Hall, Englewood Cliffs, N. J.), 1968, 39-98.
This result was generalized in
R. Messer, A. Wright, Embedding open 3-manifolds in compact 3-manifolds. Pacific J. Math. 82 (1979), no. 1, 163–177.
who found necessary and sufficient conditions for embedding in compact 3-manifolds of open 3-manifolds of the form $$ \bigcup_{n \in {\mathbb N}} M_n, $$ where for all $n$, $M_n$ is a compact submanifold with toral boundary and $M_n\subset int(M_{n+1})$.