My answer from the MO thread:
A matrix is just a list of numbers, and you're allowed to add and multiply matrices by combining those numbers in a certain way. When you talk about matrices, you're allowed to talk about things like the entry in the 3rd row and 4th column, and so forth. In this setting, matrices are useful for representing things like transition probabilities in a Markov chain, where each entry indicates the probability of transitioning from one state to another. You can do lots of interesting numerical things with matrices, and these interesting numerical things are very important because matrices show up a lot in engineering and the sciences.
In linear algebra, however, you instead talk about linear transformations, which are not (I cannot emphasize this enough) a list of numbers, although sometimes it is convenient to use a particular matrix to write down a linear transformation. The difference between a linear transformation and a matrix is not easy to grasp the first time you see it, and most people would be fine with conflating the two points of view. However, when you're given a linear transformation, you're not allowed to ask for things like the entry in its 3rd row and 4th column because questions like these depend on a choice of basis. Instead, you're only allowed to ask for things that don't depend on the basis, such as the rank, the trace, the determinant, or the set of eigenvalues. This point of view may seem unnecessarily restrictive, but it is fundamental to a deeper understanding of pure mathematics.
I consider both of Strang's books must-reads for undergraduates. These are what applied mathematics books should look like. There's a perception that applied books are merely rote procedural texts, where proofs are omitted and they basically give toolbox approaches to the material. Bad applied books do this—good ones like Strang's don't omit proofs, but they merely downplay them. They are both rich with relevant applications—that is, applications that are currently being used in research and practice by applied mathematicians and other professionals—beautifully written and conceptually careful. The Calculus book has the added bonus of being available online for free at Strang's website. Don't be fooled into spending all your money on the second edition of the calculus book—it's no different from the first, it merely contains Strang's "Highlights of Calculus" in addition, which is good but doesn't really add anything to the book's quality.
A word of warning: There are several versions of Strang's linear algebra texts. The one you really want to read is Linear Algebra And Its Applications. The other book-Introduction To Linear Algebra-is a dramatically watered down version of this book and it's missing many of Strang's wonderful sidebars and digressions. So the "non-introduction" version is the one you want.
Best Answer
Quoting an answer on Quora: