In an $n$-category, a 1-morphism $f:x\to y$ is an equivalence in the maximally weak sense if there exists $g:y\to x$ such that $gf$ and $\mathrm{id}_x$ are equivalent (in the maximally weak sense) in the $n-1$-category $\mathrm{Hom}(x,x)$ while $fg$ and $\mathrm{id}_y$ are equivalent in $\mathrm{Hom}(y,y)$. This gives an inductive definition with perhaps the most intuitive base case being given at $n=1$, where an equivalence in a $1$-category is simply an isomorphism. It is also possible to give a coinductive definition of equivalence in an $\infty$-category, where the inductive definition would never terminate; basically one ends up with an infinite binary tree of higher and higher morphisms relating a composite to an identity in both directions.
The axioms for an $n$-category in the weak sense are rather difficult to write down explicitly, to the point that it's never been done for $n>4$. This is a significant motivation for quasicategories, the topic of Lurie's book: they package together a highly complex family of axioms for $(\infty,1)$-categories, which indeed we don't even know how to express explicitly and algebraically, into the less explicit but very usable horn-lifting conditions. That said, associativity isn't too hard to understand, given the previous paragraph. Indeed, $A:f(gh)\to (fg)h$ is simply an equivalence, in the weak sense, in the hom $n-1$-category. The problem is coming up with all of the various equivalences necessary to fully axiomatize a weak $n$-category.
"Composability" and "invertibility" are not, as you've noted, really the relevant primitive notions in a quasicategory. But horn-filling accounts for all the possibilities you want. The way to make this all make sense is to consider your quasicategory as generalizing the nerve of a 2-category. Given a 2-category $\mathcal K$, its nerve has $0$-simplices the objects of $\mathcal K$ and 1-simplices the 1-morphisms; a 2-simplex with boundary
\begin{array}{ccc}
x&\xrightarrow{f}&y\\&\searrow \scriptsize{h}&\downarrow \scriptsize g\\&&z
\end{array}
is a 2-morphism $\alpha:g\circ f\to h$. Higher simplices then arise from pasting diagrams in $\mathcal K$, much as for the nerve of an ordinary category. Thus the 2-simplices in a quasicategory aren't quite what you think of when you picture a 2-morphism; if $f$ is an identity, though, then such a 2-simplex corresponds precisely to a 2-morphism $g\to h$.
With this perspective, the construction you suggest does indeed capture the notion of composition of $\sigma_1$ and $\sigma_2$. Specifically, if the edges $0\to 1$ and $1\to 2$ are degenerate, then choosing the doubly degenerate 2-simplex for the $0\to 1\to 2$ face defines a composite $\sigma_1\circ \sigma_2$ that agrees with the composite in the 2-category $\mathcal K$ in case your quasicategory is the nerve of $\mathcal K$.
As for invertibility, we can tell a similar story. Given $\sigma_1$ with, again, $0\to 1$ degenerate, one can construct an "inverse" by filling a horn with $\sigma_1$ as the $0\to 1\to 3$ face, the $0\to 1\to 2$ face double degenerate, and the $0\to 2\to 3$ face degenerate on the nondegenerate edge of $\sigma_1$. Again, in case your quasicategory is the nerve of the 2-category $\mathcal K$, this reconstructs the inverse of the 2-morphism represented by $\sigma_1$.
Your construction gives a good generalization of composition to 2-morphisms, but in fact the most natural notion of composition of 2-morphisms in a quasicategory is to compose together any three 2-morphisms that fit together into an outer horn. That is, there's no good reason, from the perspective of the quasicategory, to focus on filling horns where the $0\to 1\to 2$ face is degenerate.
On the other hand, to talk about invertibility in a quasicategory it really helps to make some edges degenerate. If we picture a 2-simplex as a 2-morphism $(g,f)\to h$, then it doesn't make sense to ask for an inverse $h\to (g,f)$. A quasicategorical way of stating formally that a quasicategory "is" an $(\infty,1)$-category is, then, that "every special outer horn has a filler", where an outer horn is special if its $0\to 1$ edge (in the case of a 0-horn) or its $n-1\to n$ edge (in the case of an $n$-horn) is an equivalence (which means it might as well be degenerate.)
Best Answer
Certainly not. The homs $\mathrm{Hom}(a,b)$ between two objects in an $\infty$-category can be modeled by topological spaces or by Kan complexes, and an equivalence of $\infty$-categories induces, in particular, homotopy equivalences between each corresponding pair of homs. In a category, these homs are, of course, mere sets, so any $\infty$-category containing any higher homotopy groups in any of its hom-spaces is not equivalent to a category (and conversely.) If this were possible, then these things wouldn't get anywhere close to as much attention as they do.
An $\infty$-category in which all morphisms are invertible is, again, equivalent to a groupoid if and only if the morphism spaces have no higher homotopy groups.
Instead, an $\infty$-category can be most strictly presented as a simplicially enriched category: the composition is associative on the nose, but you still have to carry around higher simplices in the hom-simplicial sets and think about how to compose them as well. And even then, only very special and quite complicated simplicial categories will have the property that every map between the $\infty$-categories they model arises from a functor of simplicial categories.