I'll post an answer to what I think is a reasonable question, hoping that someone more expert will improve on this answer. My apologies if this answer is too chatty.
First, a very simple reason you might hope that there is something like a cotangent complex and that it should be an object in the derived category of quasi-coherent sheaves. Given a morphism $f: X \rightarrow Y$ of schemes (over some base, which I leave implicit), you get a right exact sequence of quasi-coherent sheaves on $X$:
$$f^{*}\Omega^{1}_{Y} \rightarrow \Omega^{1}_{X} \rightarrow \Omega^{1}_{X/Y} \rightarrow 0.$$
Experience has taught us that when we have a functorial half-exact sequence, it can often be completed functorially to a long exact sequence involving 'derived functors', and in abelian contexts such a long exact sequence is usually associated to a short exact sequence (or exact triangle) of 'total derived functor' complexes, the complexes being well-defined up to quasi-isomorphism and hence objects in a derived category. This is the semi-modern point of view on derived functors, which one can learn for instance from Gelfand-Manin's Methods of Homological Algebra. Experience has also shown that not only is the cohomology of the total derived functor complex important in computations, but that the complex itself, up to quasi-isomorphism, contains strictly more information and is often easier to work with, until the very last moment when you want to compute some cohomology.
Once you've seen this work a number of times (say for global sections, for $Hom$, and for $\otimes$), one might ask if there is a total derived functor of $\Omega^{1}$, call it $\mathbb{L}$, which among other things produces an exact triangle
$$f^{*}\mathbb{L}_{Y} \rightarrow \mathbb{L}_{X} \rightarrow \mathbb{L}_{X/Y}$$
such that the long exact sequence of cohomology sheaves begins with the original right exact sequence.
If you believe that such a thing should be useful, then you might go about trying to construct it. One way to do this involves interpreting $\Omega^{1}$ as representing derivations which in turn correspond to square-zero extensions, and this is where deformation theory comes in. So one might begin to ask what 'derived square-zero extensions' should be, and you might guess that you should try to extend not just by modules but by bounded above complexes of modules. When you do this, such a square-zero extension becomes not just a commutative algebra but some kind of derived version thereof, such as a simplicial commutative algebra. In these terms, the cotangent complex $\mathbb{L}$ of a commutative algebra turns out to be nothing but K\"ahler differentials of an appropriate resolution of our commutative algebra in the world of simplicial commutative algebras. Following this idea through and figuring out how descent should work leads, after a long song and dance, to the desired theory of the cotangent complex.
Once you set this all up, it becomes clear that there was no reason to restrict oneself to classical commutative algebras in setting up algebraic geometry, but that one could have worked with simplicial commutative algebras to begin with, and this leads to `derived algebraic geometry'. In some sense, this is the natural place in which to understand the cotangent complex, and here the higher cohomology of the cotangent complex has a natural geometric interpretation.
One should also point out that Quillen's point of view on the cotangent complex was as a homology theory for commutative algebras (search for Andre-Quillen homology), which in a precise sense is an analogue of the usual homology of a topological space. This is described in the last chapter of Quillen's Homotopical Algebra and is also discussed in Goerss-Schemmerhorn's Model Categories and Simplicial Methods.
To summarise. Deformation theory is about square-zero extensions (as well as about other more general infinitesimal extensions). K\"ahler differentials corepresent derivations which in turn correspond to square-zero extensions. For each morphism of schemes $X \rightarrow Y$, there is a natural right exact sequence involving K\"ahler differentials, which it would be useful to complete to a long exact sequence. Even better,
we'd like this long exact sequence to come from an exact triangle of objects in the derived category of quasi-coherent sheaves. Realising this goal naturally leads to derived or homotopical algebraic geometry.
To answer your first precise questions:
Yes, every distinguished triangle in $D(A)$ comes from a short exact sequence. For every distinguished triangle $X \to Y \to \mathrm{Cone}(f) \stackrel{+1}\to $ there is a short exact sequence
$$ 0 \to Y \to \mathrm{Cone}(f) \to X[-1] \to 0$$
of complexes in $A$, and our distinguished triangle arises from this one by rotation.
By the same argument, every distinguished triangle in $K(A)$ comes from a short exact sequence (at least up to a rotation). However, not every short exact sequence gives rise to a distinguished triangle in $K(A)$. If
$$ 0 \to X \stackrel f \to Y \to Z \to 0$$ is a short exact sequence of complexes in $A$, then there is a natural map $\mathrm{Cone}(f) \to Z$ which in general is only a quasi-isomorphism, not a homotopy equivalence. If for example the short exact sequence splits degree-wise, then it is always a homotopy equivalence, and we get a triangle in $K(A)$.
Alright. About your question "Why $K(A)$?". You are right that homotopy equivalences are quasi-isomorphisms. So in principle one could construct $D(A)$ in "one step" by taking the category of complexes and inverting quasi-isomorphisms. But there are some technical reasons for preferring the construction via the category $K(A)$. First off, the category $K(A)$ is already triangulated, and this is easy to prove. This means that to construct $D(A)$ we are in the situation of Verdier localization: we have a triangulated category, and we localize it at the class of morphisms whose cone is in a specific thick triangulated subcategory. In particular $D(A)$ becomes triangulated. In general, localizations of categories can be complicated, and in the "one-step" construction it is not even obvious that $D(A)$ is a category, i.e. that the morphisms from one object to another form a set.
To motivate why we should care about triangles at all, note that it doesn't make sense to talk about kernels or images of morphisms in $D(A)$, so that we can't talk about exactness. Given that we want to be able to do homological algebra we need some sort of substitute for this. Triangles, encoding short exact sequences, turn out to be enough to develop much of the theory, and a posteriori one could want to axiomatize the theory only in terms of triangles. One reason we could expect the notion of a "distinguished triangle" to really be intrinsic to $D(A)$ (even though the class of distinguished triangles need to be specified in the axioms for a triangulated category) is that any triangle
$$ X \to Y \to Z \stackrel{+1}\to$$
gives rise to a long sequence of abelian groups after applying $[W,-]$ for any object $W$; this long sequence will be a long exact sequence when the triangle is distinguished, for any $W$, and this is really a very special property! A remark is that nowadays people will tell you that "stable $\infty$-categories" are for all purposes better than triangulated categories, and in a stable $\infty$-category, the class of distinguished triangles does not need to be specified in advance, so to speak: equivalent stable $\infty$-categories will have the same distinguished triangles.
Best Answer
Here is an example mentioned in passing by user ali's answer, but I think it is cute (and powerful) enough to be worth fleshing out the details.
Lifting from characteristic $p$ to characteristic zero
In short, studying a geometric object (say, a scheme) $X$ in characteristic $p$ often involves lifting it to characteristic zero. For example, if $X$ is a smooth projective variety over $\mathbf{F}_p$, we may try to find a (flat) lift $\mathcal{X}$ over the $p$-adic numbers $\mathbf{Z}_p$. Now, $\mathbf{Z}_p$ embeds into $\mathbf{C}$ (in some completely noncanonical way), and we can apply powerful methods such as Hodge theory to the complex manifold underlying $\mathcal{X}_\mathbf{C}$.
Now, recall that $$ \mathbf{Z}_p = \varprojlim_n \mathbf{Z}/p^{n+1}. $$ Thus lifting $X_0=X$ over $\mathbf{Z}_p$ involves finding compatible liftings $X_n$ over $\mathbf{Z}/p^{n+1}$ for all $n$. The system $\mathfrak{X} = \{X_n\}$ (or its inductive limit in locally ringed spaces) is a "$p$-adic formal scheme," and the next step involves checking that it is algebraizable, i.e. that it comes from an actual scheme $\mathcal{X}/\mathbf{Z}_p$ by the obvious "formal completion" functor.
Now the first step, finding the successive liftings $\{X_n\}$, is completely controlled by deformation theory. In our situation, it says the following:
If $X_0$ is a scheme over $\mathbf{F}_p$, and $X_n$ is a flat lifting of $X_0$ over $\mathbf{Z}/p^{n+1}$, there there exists an obstruction class $$ {\rm obs}(X_n, \mathbf{Z}/p^{n+2}) \in {\rm Ext}^2(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0}) = {\rm Hom}_{D(X_0)}(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0}[2]), $$ which vanishes if and only if there exists a flat lifting $X_{n+1}$ of $X_n$ over $\mathbf{Z}/p^{n+2}$. It is functorial in the sense that for $f_n\colon X_n\to Y_n$ lifting $f_0\colon X_0\to Y_0$ we have a commutative square $$\require{AMScd} \begin{CD} \mathbf{L}_{Y_0/\mathbf{F}_p} @>>> \mathcal{O}_{Y_0}[2]\\ @VVV @VVV\\ Rf_{0, *}\mathbf{L}_{X_0/\mathbf{F}_p} @>>> Rf_{0, *}\mathcal{O}_{X_0}[2] \end{CD}$$
In case the obstruction class vanishes, the set of isomorphism classes of such liftings $X_{n+1}$ is in a natural way a torsor under $$ {\rm Ext}^1(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0}) = {\rm Hom}_{D(X_0)}(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0}[1]). $$
The group of automorphisms of any lifting $X_{n+1}$ restricting to the identity on $X_n$ is naturally isomorphic to $$ {\rm Hom}(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0}). $$
There is a similar story for lifting morphisms $f_0\colon X_0\to Y_0$.
So if you can show that ${\rm Ext}^2(\mathbf{L}_{X_0/\mathbf{F}_p}, \mathcal{O}_{X_0})$ vanishes, then you know that $X_0$ admits a formal $p$-adic lifting $\mathfrak{X}$. For example, if $X_0$ is a K3 surface, then this group can be identified with the space of global vector fields on $X_0$, and its vanishing is a difficult theorem due to Rudakov and Shafarevich. (And the fact that there is an algebraizable formal lifting, i.e. that an ample line bundle can be lifted to all $X_n$'s for a good choice of $\mathfrak{X}$, was shown later by Deligne.)
Perfect schemes and Witt vectors
Recall that for every perfect field $k$ of characteristic $p>0$ there exists a unique complete discrete valuation ring $W(k)$ (its ring of Witt vectors) with residue field $k$ whose maximal ideal is generated by $p$. It is a functor of $k$, and we have $W(k) \simeq k^{\mathbf{N}}$ as functors into sets. The addition and multiplication laws on $k^{\mathbf{N}}$ obtained this way are given by complicated universal formulas, e.g. $$ (x_0, x_1, \ldots) + (y_0, y_1, \ldots) = (x_0 + y_0, x_1 + y_1 - \sum_{0<i<p} \frac 1 p \binom p i x_0^i y_0^{p-i}, \ldots). $$ We define $W_n(k) = W(k)/p^n$ and call these Witt vectors of length $n$.
For example, $W(\mathbf{F}_p) = \mathbf{Z}_p$, $W_n(\mathbf{F}_p) = \mathbf{Z}/p^n$.
In fact, the above can be defined for any ring $R$. If $R$ is a perfect $\mathbf{F}_p$-algebra, meaning that its Frobenius $$ F_R \colon R\to R, \quad F_R(x) = x^p $$ is an isomorphism, then $W(R)$ is a flat lifting of $R$ over $W(\mathbf{F}_p) = \mathbf{Z}_p$.
Here is a beautiful argument (I think due to Bhargav Bhatt) employing the cotangent complex to show the existence of Witt vectors for perfect rings (or schemes) without using any strange-looking universal formulas for addition and multiplication.
The above implies that $\mathfrak{X}$ is a functor of $X$, denoted $W(X)$. It is not difficult to prove that it indeed coincides with the Witt vectors.
Proof. Consider the cotangent complex $\mathbf{L}_{X_0/\mathbf{F}_p}$ and the map $$ F_X^* \colon \mathbf{L}_{X_0/\mathbf{F}_p}\to F_{X, *} \mathbf{L}_{X_0/\mathbf{F}_p} $$ induced by the absolute Frobenius $F_X\colon X\to X$. Since $F_X$ is an isomorphism, the map $F_X^*$ is an isomorphism too. The complex $\mathbf{L}_{X_0/\mathbf{F}_p}$ is defined by locally resolving $\mathcal{O}_X$ by free $\mathbf{F}_p$-algebras and considering their Kaehler differentials. And $F_A$ acts as zero on $\Omega^1_{A/\mathbf{F}_p}$ for every $\mathbf{F}_p$-algebra $A$: $$ F_A^*(dx) = dF_A(x) = dx^p = px^{p-1} dx = 0. $$ Therefore the map $F_X^*$ above is the zero map. Since it is also an isomorphism, we conclude that $\mathbf{L}_{X_0/\mathbf{F}_p} = 0$!
Now by deformation theory, the obstructions to lifting lie in the zero group (and hence the successive liftings exist), the isomorphism classes of different successive liftings are permuted by the zero group (and hence the liftings are unique), and their automorphism groups are trivial (so the liftings are unique up to a unique isomorphism). Similarly, one handles the lifting of morphisms. $\square$