Here is an elementary and intuitive explanation. The fiber of a map is locally a tensor product: if $X=\text{Spec} S$ and $Y=\text{Spec} R$ and the ring map is $R \to S$, then the fiber at a point $p \in Y$ is the Spec of $R_p/pR_p \otimes_R S$.
Flatness is exactly the condition that makes tensor products behave like a dream (almost by definition), it preserves a lot of useful structures. Many algebraic results with geometric consequences go like this: let $(P)$ be a reasonable property and $f: R\to S$ a flat local homomorphism. Then $S$ satisfies $(P)$ if and only if $R$ and the fiber at the closed point satisfy $(P)$ (these are called Grothendieck localization problem).
I am not a historian, but I suspect that was how flatness arised: people wanted certain nice things to be true, and were naturally lead to flatness (see BCnrd's comment below for the precise history).
Everything holds for a smooth quasiprojective $X$ as long as there exists a $\mathbb{C}^*$ action so that
$$
\lim_{t \to 0} t \cdot x
$$
exists for every $x \in X$. This is guaranteed when $X$ is projective but holds more generally. The theory really depends on a local analysis of torus actions at fixed points so as long as you have enough fixed points for the limits to exist (and $X$ is smooth), everything holds.
In the projective case, say you have a rank $r$ torus $T = (\mathbb{C}^*)^r$ acting on $X$, the one parameter subgroups $\mathbb{C}^* \cong T' \subset T$ form a lattice $N$ of rank $r$. For a projective $X$, any general enough $z \in N$ gives you a one parameter subgroup with the properties required to get a BB decomposition. Furthermore, scaling $z$ by a positive multiple does not change the decomposition. So you get a cone decomposition of $N \otimes_\mathbb{Z} \mathbb{R}$ that determine your BB decomposition.
As you vary within the interior of a top dimensional cone, the induced BB decomposition stays the same, and then when you cross the faces between the top dimensional cones (which correspond to degenerate choices of one parameter subgroup that do not give you enough independent conditions for the fixed points of $T'$ to be the same as those of $T$) you change the decomposition. Of course within each cone we need to pick a point that is scaled large enough.
So in the quasiprojective case, the difference is that now you have to stay within certain top dimensional cones where the limit above always exists, while the other regions of $N \otimes_\mathbb{Z} \mathbb{R}$ correspond to one parameter subgroups whose limit would be outside of the variety, say in an appropriate projective compactification.
Then once you have a decomposition into locally closed affine spaces you get the usual theorems about the cohomology vanishing in odd degree and being generated in even degree by the closures of the appropriate dimension affine cells, etc.
As an example consider $\operatorname{Hilb}^n(\mathbb{C}^2)$ which I will denote $H^n$ for convenience. $(\mathbb{C}^*)^2 = T$ acts on $\mathbb{C}^2$ in the obvious way and thus on $H^n$. The points of $H^n$ are ideals $I$ in $\mathbb{C}[x,y]$ so that $\dim_\mathbb{C}\mathbb{C}[x,y]/I = n$. The torus fixed points of this action correspond to the monomial ideals. If you pick a subtorus $(t^p,t^q)$ with $p,q > 0$, this gives a monomial order $w$ on $\mathbb{C}[x,y]$ and
$$
\lim_{t \to 0} (t^p,t^q)\cdot I = in_w(I)
$$
the initial ideal with respect to this ordering, which for general orderings is always a monomial ideal in $H^n$ and so gives a BB decomposition. The special orderings where the initial ideal is not always a monomial ideal are the faces of the cones. However, if we pick $p,q < 0$, then this limit can be an ideal supported at infinity in $\mathbb{P}^2$ and so we do not get a BB decomposition of $H^n$.
See also this MO question: Cell decomposition for a variety not necessarily complete?.
Best Answer
It is a theorem of Sumihiro (Equivariant completion, Corollary 2) that a normal variety over an algebraically closed field with an action of a torus is covered by invariant affine open subsets.
Here, the normality hypothesis is necessary : the conclusion does not hold for the action of $\mathbb{G}_m$ on $\mathbb{P}^1$ with $0$ and $\infty$ glued together transversally. The hypothesis that the group is a torus is also necessary. Indeed, the statement already fails for SL(2), even for a proper action. There is an example in Białynicki-Birula and Święcicka's paper "On complete orbit spaces of SL(2) actions II".