I'll assume that the quiver $Q$ has finitely many vertices and arrows.
Then even if $Q$ has oriented cycles, it is it is still true that the path algebra $\mathbb{C}Q$ determines the quiver $Q$.
(1) In the case of a quiver with no oriented cycles, a quick way to recover the quiver is as follows:
There is a simple module $S_i$ associated with each vertex $i$, and the number of arrows from vertex $i$ to vertex $j$ is $\dim_{\mathbb{C}}\operatorname{Ext}^1_{\mathbb{C}Q}(S_i,S_j)$. [I'm not sure that this is the proof that Derksen was asking for, as the book doesn't seem to assume knowledge of $\operatorname{Ext}$.]
So knowing the simple $\mathbb{C}Q$-modules and the extensions between them lets you recover the quiver.
(2) If the quiver has oriented cycles but no loops (arrows with the target equal to source) then there are more simple modules, but the obvious simples associated to the vertices are the only $1$-dimensional simples, and the same method of recovering the quiver works, if you only consider the $1$-dimensional simples.
(3) If the quiver has loops, then there are more $1$-dimensional simples (consider the representation with $\mathbb{C}$ at vertex $i$, zero at every other vertex, with the loops at vertex $i$ acting by multiplication by arbitrary scalars). But the same method work if we can pick out one $1$-dimensional simple module for each vertex.
There may be a simpler method, but one way to do this is to consider the abelianization of $\mathbb{C}Q$. This is a product of polynomial algebras $\mathbb{C}[x_1,\dots,x_{r_i}]$, one for each vertex $i$, where $r_i$ is the number of loops at vertex $i$. So it has one primitive idempotent for each vertex, and for each of these idempotents we can choose any $1$-dimensional simple module (it doesn't matter which) that is not annihilated by that idempotent. As before, the quiver can then be recovered by considering $\operatorname{Ext}^1_{\mathbb{C}Q}$ between these simple modules.
Best Answer
Morita Equivalence This is a supplement to the aspect of quiver representations mentioned in Alistair's answer. Every associative finite dimensional $k$-algebra $A$ is Morita equivalent to a path algebra $kQ/I$ (this is another Gabriel's theorem). In particular, you have a very nice equivalence (so nice that the functors giving such equivalence are given by tensoring finitely-generated projective bimodules) of abelian categories $A\text{-Mod}$ and $kQ/I\text{-Mod}$. So basically (almost) everything you want to know about representations of $A$ ($A$-modules) can be learnt from studying $kQ/I$-modules. And studying quiver representation is arguably much easier because path algebras are basic, meaning all simple modules are one-dimensional. This is equivalent to saying all projective indecomposable are only of multiplicity $1$ in $kQ/I$, one can say that this makes the homological behaviour of the modules much easier to study. In particular, many things can be done combinatorially.
Auslander-Reiten Theory This part does not relate directly to "why quiver representation", but to "why quivers". It turns out we can treat abelian categories (or in fact functorially finite categories) pretty much the same way as we treat an algebra: you can talk about irreducible maps. In particular, there is a combinatorial gadget called the Auslander-Reiten quiver, where vertices are in correspondence with indecomposable $A$-module and arrows are given by irreducible maps. In such a way, one can "visualise" the category nicely with a very nice form of quivers. And surprisingly, the form of quiver appearing in this construction also follows Dynkin classification.
Cluster Theory One of the most exciting developments in representation theory in the last decade is the cluster theory introduced by Fomin and Zelevinsky. The centre of the theory is an algebra called the cluster algebra, which is some sort of dual to the picture we see in the theory of Lie algebras (I do not know the exact argument to this). But the algebraic setting for clusters theory is pretty difficult to work with sometimes, and it turns out we can use an operation on quivers called a quiver mutation to substitute basis elements of the cluster algebra. Now people "categorify" this setting (which is in fact an incarnation of the Auslander-Reiten theory I mentioned above) and found out that we can use the derived category of the module category of the path algebra $kQ$ to study properties of cluster algebras.
Hall Algebras There is one construction of algebra called the Hall algebra of an abelian category. Ringel proved the amazing theorem in the 90s that if you take a Dynkin quiver $Q$ and consider the module category $kQ\text{-mod}$, then take the Hall algebra $H_Q$ of $kQ\text{-mod}$, it turns out $H_Q$ is isomorphic as an Hopf algebra to one half of the quantum group of the Lie algebra of type $Q$. I.e. studying quiver representations allow us to dig out more unknown properties of quantum group.
Quiver Varieties & Geometric Representation Theory (This is the impression I have got. Please correct me if I am wrong) If you recall from the proof of Gabriel's theorem on the classification of finite-type (unquotiented) path algebras, you will see there is some action of general linear group on the quiver. Nakajima extended this idea and developed a whole new approach for doing representation theory via geometric methods, using the so called Nakajima quiver variety.