Here is an elementary and intuitive explanation. The fiber of a map is locally a tensor product: if $X=\text{Spec} S$ and $Y=\text{Spec} R$ and the ring map is $R \to S$, then the fiber at a point $p \in Y$ is the Spec of $R_p/pR_p \otimes_R S$.
Flatness is exactly the condition that makes tensor products behave like a dream (almost by definition), it preserves a lot of useful structures. Many algebraic results with geometric consequences go like this: let $(P)$ be a reasonable property and $f: R\to S$ a flat local homomorphism. Then $S$ satisfies $(P)$ if and only if $R$ and the fiber at the closed point satisfy $(P)$ (these are called Grothendieck localization problem).
I am not a historian, but I suspect that was how flatness arised: people wanted certain nice things to be true, and were naturally lead to flatness (see BCnrd's comment below for the precise history).
Instead of trying to say what flatness in analytic geometry means I'll give you some street-fighting tricks for recognizing whether a morphism of analytic spaces ( not necessarily reduced) $f:X\to Y $ is , or has a chance to be, flat.
a) A flat map is always open. . Hence, contraspositely, the embedding
$\lbrace 0\rbrace \hookrightarrow \mathbb C$ is not flat. More generally, the embedding of a closed (and not open !) subspace $X \hookrightarrow Y$ is never flat.
b) An open map need not be flat: think of the open map $ Spec(\mathbb C) \to Spec(\mathbb C[\epsilon ]) $ from the reduced point to the double point, which is open but not flat.
[It is not flat because the $\mathbb C[\epsilon]$- algebra $\mathbb C=\mathbb C[\epsilon]/(\epsilon)$ is not flat : recall that a quotient ring $A/I$ can only be flat over $A$ if $I=I^2$ and here
$I=(\epsilon) \neq I^2=(\epsilon)^2=(0)$]
An example with both spaces reduced is the normalization [see g) below] $f:X^{nor} \to X $ of the cusp $X\subset \mathbb C^2$ given by the equation $y^2=x^3$ .That normalization is a homeomorphism and so certainly open, but it is not flat : this results either from g) or from h) below.
c) Given the morphism $f:X\to Y $ , consider the following property:
$\forall x \in X, \quad dim_x(X)=dim_{f(x)} (Y) +dim_x(f^{-1}(f(x)))$ $\quad (DIM) $
We then have: $ f \; \text {flat} \Rightarrow f \;\text { satisfies } (DIM)$
For example a (non-trivial) blowup is not flat.
d) For a morphism $f:X\to Y $ between connected holomorphic manifolds we have:
$$f \text { is flat} \iff f \text { is open} \quad \iff (DIM) \;\text {holds}$$
For example a submersion is flat, since it is open.
e) Given two complex spaces $X,Y$ the projection $X\times Y\to X$ is flat ( Not trivial: recall that open doesn't imply flat!).
f) flatness is preserved by base change.
g) The normalization $f:X^{nor} \to X $ of a non-normal space is never flat.
For example if $X\subset \mathbb C^2$ is the cusp $y^2=x^3$, the normalization morphism $\mathbb C\to X: t\mapsto (t^2,t^3)$ is not flat.
h) Given a finite morphism $f:X\to Y $, each $y\in Y$ has a fiber $X(y) \subset X$ and for $x\in X$ we can define $\mu (x)= dim_{\mathbb C} (\mathbb C \otimes_{\mathcal O_{Y,y}} \mathcal O_{X,x})$ . Now for $ y\in Y$ we put $\; \nu (y)= \Sigma_{x\in X(y)} \mu(x)$ and we obtain :
$$f \; \text{ flat} \iff \nu :Y\to \mathbb N \text { locally constant}$$
[Of course for connected $Y$, locally constant = constant]
Bibliography:
A. Douady, Flatness and Privilege, L'Enseignement Mathématique, Vol.14 (1968)
G.Fischer, Complex Analytic Geometry, Springer LNM 538, 1976
Best Answer
(1) A family of hypersurfaces of degree $d$ over an affine scheme $Spec\ R$ means the following: it is a closed subscheme of $\mathbb P^n_R$ given by a homogeneous polynomial $f(x_0,\dots,x_n)$ of degree $d$ satisfying the following condition:
Now, let $m$ be a maximal ideal, so that $k$ is a field, and look at the graded ring $k[x_0,\dots,x_n]/(f)$. For each $a\ge d$, the degree-$a$ part is a vector space of dimension $\binom{a+n}{n}-\binom{a-d+n}{n}$. Thus, some $\binom{a-d+n}{n}$ monomials can be written as linear combinations, with coefficients in $k$, of the remaining monomials. This is done by solving a system of linear equations obtained by setting $x^m f=0$, where $x^m$ are monomials of degree $a-d$.
Now, consider the same system of linear equations with coefficients in $R$. For each of the $\binom{a-d+n}{n}$ monomials as above you get a principal minor $M$ of your matrix, and the reduction of $\det M$ in $k$ is not zero. Thus, over an open set $Spec\ R[1/\det M]$, this determinant is invertible, and the monomial can be eliminated.
Thus, over an open neighborhood $Spec\ A$ of the point $[m]\in Spec\ R$, the degree-$a$ part of the ring $A[x_0,\dots,x_n]/(f)$ is a free $A$-module. Recalling how $Proj$ is covered by $Spec$'s and that a free module is flat, this implies that $Proj\ S[x_0,\dots,x_n]/(f)$ is flat over $Spec\ S$. (We will assume $R$ and so $A$ to be Noetherian here for simplicity.)
This is the main trick for proving flatness over a non-reduced base: you prove freeness instead. For a finitely generated module over a Noetherian ring, flatness and freeness are equivalent. So for a projective morphism $f:X\to Y$ a coherent sheaf $F$ on $X$ is flat over $Y$ iff the sheaves $f_* F(a)$ on $X$ are locally free for $a\gg0$.
In (2), you are mistaken about Hartshorne: Theorem III.12.11 (Cohomology and Base Change) has no assumption for the base to be reduced. So if $H^i(X_y,F_y)=0$ for $i>0$ then $f_*F$ is locally free, for any (Noetherian) base $Y$ and coherent sheaf $F$ on $X$, flat over $Y$.
For higher direct images, $H^i(X_y,F_y)=0$ for $i\ge i_0$ implies that $R^{i_0}f_*F=0$. But you can have $H^i(X_y,F_y)=0$ for $i> i_0$ and $H^{i_0}(X_y,F_y)$ non-constant, and still have $R^{i_0}f_*F=0$ (compare the Poincare line bundle on $A\times A^t$, as in Fourier-Mukai).