It is impossible to produce an example of a finitely generated flat $R$-module that is not projective when $R$ is an integral domain. See: Cartier, "Questions de rationalité des diviseurs en géométrie algébrique," here, Appendice, Lemme 5, p. 249. Also see Bourbaki Algèbre Chapitre X (Algèbre Homologique, "AH") X.169 Exercise Sect. 1, No. 13. I also sketch an alternate proof that there are no such examples for $R$ an integral domain below.
Observe that, for finitely generated $R$-modules $M$, being locally free in the weaker sense is equivalent to being flat [Bourbaki, AC II.3.4 Pr. 15, combined with AH X.169 Exercise Sect. 1, No. 14(c).]. ($R$ doesn't have to be noetherian for this, though many books seem to assume it.)
There's a concrete way to interpret projectivity for finitely generated flat modules. We begin by translating Bourbaki's criterion into the language of invariant factors. For any finitely generated flat $R$-module $M$ and any nonnegative integer $n$, the $n$-th invariant factor $I_n(M)$ is the annihilator of the $n$-th exterior power of $M$.
Lemma. (Bourbaki's criterion) A finitely generated flat $R$-module $M$ is projective if and only if, for any nonnegative integer $n$, the set $V(I_n(M))$ is open in $\mathrm{Spec}(R)$.
This openness translates to finite generation.
Proposition. If $M$ is a finitely generated flat $R$-module, then $M$ is projective iff its invariant factors are finitely generated.
Corollary. The following conditions are equivalent for a ring $R$: (1) Every flat cyclic $R$-module is projective. (2) Every finitely generated flat $R$-module is projective.
Corollary. Over an integral domain $R$, every finitely generated flat $R$-module is projective.
Corollary. A flat ideal $I$ of $R$ is projective iff its annihilator is finitely generated.
Example. Let me try to give an example of a principal ideal of a ring $R$ that is locally free in the weak sense but not projective. Of course my point is not the nature of this counterexample itself, but rather the way in which one uses the criteria above to produce it.
Let $S:=\bigoplus_{n=1}^{\infty}\mathbf{F}_2$, and let $R=\mathbf{Z}[S]$. (The elements of $R$ are thus expressions $\ell+s$, where $\ell\in\mathbf{Z}$ and $s=(s_1,s_2,\dots)$ of elements of $\mathbf{F}_2$ that eventually stabilize at $0$.) Consider the ideal $I=(2+0)$.
I first claim that for any prime ideal $\mathfrak{p}\in\mathrm{Spec}(R)$, the $R_{\mathfrak{p}}$-module $I_{\mathfrak{p}}$ is free of rank $0$ or $1$. There are three cases: (1) If $x\notin\mathfrak{p}$, then $I_{\mathfrak{p}}=R_{\mathfrak{p}}$. (2) If $x\in\mathfrak{p}$ and $\mathfrak{p}$ does not contain $S$, then $I_{\mathfrak{p}}=0$. (3) Finally, if both $x\in\mathfrak{p}$ and $S\subset\mathfrak{p}$, then $I_{\mathfrak{p}}$ is a principal ideal of $R_{\mathfrak{p}}$ with trivial annihilator.
It remains to show that $I$ is not projective as an $R$-module. But its annihilator is $S$, which is not finitely generated over $R$.
[This answer was reorganized on the recommendation of Pete Clark.]
For any commutative ring $R$ with 1, any projective $R$-module of constant finite rank is finitely generated. This is the content of Exercises I.2.13 and I.2.14 of Weibel's $K$-book.
The argument goes as follows (all the relevant hints are in Weibel's book), I hope I did not introduce any tacit finiteness assumptions.
1) In the constant rank one case, we have the dual $\check{P}=\operatorname{Hom}_R(P,R)$ and the evaluation map $\operatorname{ev}:\check{P}\otimes_R P\to R$ whose image $\tau_P=\operatorname{im}\left(\operatorname{ev}:\check{P}\otimes_RP\to R\right)$ is called the trace. By a result of Kaplansky, $P_{\mathfrak{p}}$ is free for any prime ideal $\mathfrak{p}\subseteq R$, and we can choose a generator $x\in P_{\mathfrak{p}}$. Then $\check{x}\otimes x\mapsto 1$ under $\check{P_{\mathfrak{p}}}\otimes_{R_{\mathfrak{p}}}P_{\mathfrak{p}}\to R_{\mathfrak{p}}$. In particular, $\tau_P\not\subseteq\mathfrak{p}$ since otherwise $P\otimes_R R/\mathfrak{p}=0$ contradicts the constant rank one assumption (as pointed out in the comments of Owen Biesel). Since $\tau_P$ is an ideal, it contains $1$ and we can write $1=\sum f_i(x_i)$. For each $\mathfrak{p}$, some $x_i$ will be a generator of $P_{\mathfrak{p}}$, so the $x_i$ are generators for $P$.
2) If $P$ has constant rank $r$, then $\bigwedge^r P$ has constant rank one and by Step 1 has finitely many generators. These are of the form $\sum a_i y_{1i}\wedge\cdots\wedge y_{ri}$ and the (finitely many) $y_{ij}$ appearing generate $P$.
Best Answer
For question two the example that is given most frequently seems to be that of the ring $R$ of continuous real valued functions on $[0,1]$ and the ideal of all functions $f$ which vanish on some interval $[0,\epsilon(f)]$ where $\epsilon(f)\in (0,1)$. This ideal is countably generated and projective but not a direct sum of finitely generated submodules. You might also want to take a look at the article "When every projective module is a direct sum of finitely generated modules" by W. McGovern, G. Puninski and P. Rothmaler.