Some of the many (semi)standard references are below (with no claims to completeness or representativeness, if that's a word -- just the first references that came to mind). My feeling is the subject is still very much in its infancy however, for example one would like to know the standard package of nonabelian Hodge theory results for singular curves (geometry of Higgs bundles and local systems, Hitchin fibration, its self-duality etc) and there are partial results but no complete picture as far as I know.
Caporaso, Lucia A compactification of the universal Picard variety over the moduli space of stable curves. J. Amer. Math. Soc. 7 (1994), no. 3, 589--660.
Pandharipande, Rahul A compactification over $\overline {M}_g$ of the universal moduli space of slope-semistable vector bundles. J. Amer. Math. Soc. 9 (1996), no. 2, 425--471.
Seshadri, C. S. Moduli spaces of torsion free sheaves on nodal curves and generalisations. I. Moduli spaces and vector bundles, 484--505, London Math. Soc. Lecture Note Ser., 359, Cambridge Univ. Press, Cambridge, 2009.
(and earlier papers of his)
arXiv:1001.3868 Title: Autoduality of compactified Jacobians for curves with plane singularities
Authors: D.Arinkin
--see this reference for refs to the vast literature by Altman-Kleiman and Esteves-Kleiman on compactified Jacobians
Kausz, Ivan A Gieseker type degeneration of moduli stacks of vector bundles on curves. Trans. Amer. Math. Soc. 357 (2005), no. 12, 4897--4955 (electronic).
Schmitt, Alexander H. W. Singular principal $G$-bundles on nodal curves. J. Eur. Math. Soc. (JEMS) 7 (2005), no. 2, 215--251.
(and earlier papers of his)
Best Answer
Dear Mohammad, there is a rather elementary book Introduction to Moduli Problems and Orbit spaces by P.E. Newstead which will explain to you why stability is important, give you lots of examples (Chapter 4 is devoted to them) and which ends with a whole chapter (Chapter 5) called Vector bundles over a curve. It was written by an extremely competent expert and deliberately maintained at a quite elementary level. The author explains in the preface that his notes are an introduction to Mumford's Geometric Invariant Theory in the language of classical algebraic geometry, deliberately eschewing schemes.
On the subject of holomorphic bundles over $\mathbb P^n(\mathbb C) $ you may check Okonek, Schneider and Spindler's monograph Vector Bundles on Complex Projective Spaces, written in the language of holomorphic manofolds (the results are the same as in algebraic geometry thanks to Serre's GAGA principle).
I'd also like to mention Atiyah's classic Vector bundles over an elliptic curve published in 1957, which I still find quite instructive despite its venerable age.
And finally I should also mention the articles on moduli of vector bundles over curves written by the brilliant Indian school around the Tata Institute: M.S.Narasimhan, Seshadri, Ramanan, Nori, ...