I think there are two distinct ways that one could interpret this question, so I'll try my hand at answering both.
Interpretation 1: I am learning the standard formulation of quantum mechanics and solving problems like the particle in a box. I am comfortable performing all of these calculations, but I do not understand why I have to know about Hilbert spaces or linear algebra.
This one is pretty straightforward. If you can add things together, multiply them by constants, and take inner products, then you're essentially$^\dagger$ working with a Hilbert space. The wavefunctions which you are comfortable solving for are elements of such a space, and the self-adjoint operators which represent observables are linear maps from one element of the space to another.
Linear algebra is just the study of vector spaces and linear maps between them, so it is in particular the backdrop of all of the calculations you are performing. When you solve an eigenvalue equation like the Schrodinger equation, you're doing linear algebra. When you expand a generic state as a superposition of eigenstates of some observable, you're doing linear algebra. When you are confident that such a set of eigenfunctions even exists and that the corresponding eigenvalues are real, it is because you have learned the spectral theorem for self-adjoint operators, which is a central result in (you guessed it) linear algebra (or functional analysis, which is essentially linear algebra in infinite dimensional spaces).
In that sense, it's not so much that linear algebra is useful in the standard formulation of quantum mechanics; its that the standard formulation of quantum mechanics is linear algebra, whether you'd like to call it that or not. To be sure, particular techniques, theorems, and general mindsets from linear algebra can be extremely useful for performing calculations, making up models, etc - but the fact remains that no matter how you slice it, what you are doing is linear algebra on a Hilbert space.
$^\dagger$Actually this describes what's called a pre-Hilbert space. To be a full Hilbert space, there's an additional technical requirement called completeness. Loosely, this means that sequences which "look like" they should converge actually do. This is important whenever you use a limit, which appears when you differentiate (e.g. the $\frac{d}{dt}$ in the Schrodinger equation) and whenever you expand a wavefunction as an infinite series of eigenvectors of some observable.
Interpretation 2: I understand that the formulation of quantum mechanics that I'm currently learning is based around the Hilbert space (i.e. a vector space with an inner product) as a central concept, but I don't understand why such a construct provides the correct description of nature.
This is a much deeper question. At the deepest level, a physical theory is nothing more or less than a mechanism for assigning probabilities to the possible outcomes of measurements. The standard formulation of quantum mechanics accomplishes this in a somewhat peculiar way, wherein we devise a correspondence between measurable properties of the system and linear maps on some Hilbert space (and then proceed as you have learned).
This approach works, as has been shown by many thousands of experiments over the past hundred years, but it's far from obvious why this is the right path to take. Some insight may be obtained from the algebraic formulation of quantum mechanics, wherein the central object under consideration is the so-called algebra of observables, whose elements represent the various measurable properties of a given system.
On one hand, this is very nice - we're working and referring directly to things we intend to measure, and in many cases it's even possible to obtain this quantum algebra of observables by suitably fiddling with a corresponding classical algebra of observables (though I should say, the latter is a different variety of algebra). The downside is that this formulation of quantum mechanics is very abstract and very sophisticated - so much so that I would be willing to bet that the substantial majority of working physicists are at best only tangentially aware of its existence.
Luckily, for those of us who are not interested in studying $C^*$-algebras until our eyes bleed, there is an alternative to this heavy mathematical abstraction. According to the Gelfand-Naimark theorem, any such algebra of observables can be concretely realized as operators on some Hilbert space. In that way we are led back to the standard formulation of quantum mechanics, but with a fresh perspective: the seemingly arbitrary choice to model a quantum system around a Hilbert space is a choice born not of necessity but rather of convenience, because it provides a concrete realization of what would otherwise be a terribly abstract description of nature.
Best Answer
I depends on the book you've chosen to read. But usually some basics in Calculus, Linear Algebra, Differential equations and Probability theory is enough. For example, if you start with Griffiths' Introduction to Quantum Mechanics, the author kindly provides you with the review of Linear Algebra in the Appendix as well as with some basic tips on probability theory in the beginning of the first Chapter. In order to solve Schrödinger equation (which is (partial) differential equation) you, of course, need to know the basics of Differential equations. Also, some special functions (like Legendre polynomials, Spherical Harmonics, etc) will pop up in due course. But, again, in introductory book, such as Griffiths' book, these things are explained in detail, so there should be no problems for you if you're careful reader. This book is one of the best to start with.